NVIDIA / NVIDIA/TensorRT-Edge-LLM

fix: Jetson Orin + JetPack 6.2 (CUDA 12.6, listed as Compatible platform) fails to build with ENABLE_CUTE_DSL=ALL — missing cudaLibrary_t compat shim in cuteDslF16MoeRunner.h

Open Beginner friendly
#183 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
563
Forks
135
Avg merge
14h 13m
Merged PRs (30d)
1

Description

Describe the bug

The documented build flow for Jetson Orin + JetPack 6.2+ — a platform listed
in the Official Support Matrix
as Compatible ("expected to work with the stated constraints") — fails to
compile in 0.10.0:

In file included from cpp/kernels/cuteDSLArtifact/aarch64/sm_87/include/cutedsl_f16_moe_all.h:3,
                 from cpp/kernels/moe/f16_cutedsl/cuteDslF16MoeRunner.h:28,
                 from cpp/kernels/moe/f16_cutedsl/cuteDslF16MoeRunner.cpp:20:
cpp/kernels/cuteDSLArtifact/aarch64/sm_87/include/f16_moe_ampere_grouped_fp16.h:21:5:
error: 'cudaLibrary_t' does not name a type; did you mean 'cudaArray_t'?
   21 |     cudaLibrary_t module;

Impact: blocker — the support matrix promises Jetson Orin / JetPack 6.2+ /
CUDA 12.6 works, and the Installation guide gives a dedicated "JetPack 6.2+
Orin" CMake command that includes -DENABLE_CUTE_DSL=ALL ("All commands
enable CuTe DSL kernels because Qwen3.5 and several other model paths require
them"). Following that documented command does not build.

Root cause: cudaLibrary_t is a CUDA runtime type introduced in CUDA
12.8, but AOT-generated CuTe DSL artifact headers use it unconditionally. All
other CuTe DSL runner headers compensate with a compat typedef
(typedef CUlibrary cudaLibrary_t + inline cudaLibraryUnload) guarded by
TRT_EDGELLM_CUDA_LIBRARY_T_COMPAT — which the root CMakeLists.txt
auto-defines for EMBEDDED_TARGET=jetson-orin (see comment at
CMakeLists.txt:92-96). cuteDslF16MoeRunner.h is the only runner header
missing this block:

Header Compat block
cpp/kernels/gdnKernels/cuteDslGDNRunner.h
cpp/kernels/mamba/cuteDslSSDRunner.h
cpp/kernels/moe/nvfp4_cutedsl/cuteDslNvfp4MoeRunner.h
cpp/kernels/talkerMLPKernels/cuteDslGemmRunner.h
cpp/plugins/int4GroupwiseGemmPluginV2/cuteDslInt4Gemm.cpp / cuteDslInt4Gemv.cpp
cpp/kernels/moe/f16_cutedsl/cuteDslF16MoeRunner.h missing

Related gap: kernelSrcs/cuteDSLPrebuilt/ only ships cuda13 tarballs
(e.g. cutedsl_aarch64_sm_87_cuda13.tar.gz), so CUDA 12.6 Orin users must
generate artifacts locally via build_cutedsl.py — the failure reproduces
with a freshly generated cuda12/sm_87 artifact, so it is independent of
artifact provenance.

Proposed fix: add the identical compat block to cuteDslF16MoeRunner.h.
I have a verified patch (full build succeeds, binaries produced) and can send
a PR if approved.

Steps/Code to reproduce bug
  1. Generate a local CUDA 12 / sm_87 CuTe DSL artifact (no cuda12 prebuilt
    tarball ships in the repo):
python kernelSrcs/build_cutedsl.py --kernels ALL --gpu_arch sm_87 --cuda-version 12
  1. Configure and build with the exact "JetPack 6.2+ Orin" command from the
    Installation guide:

Build configuration:

mkdir -p build && cd build
cmake .. \
    -DCMAKE_BUILD_TYPE=Release \
    -DTRT_PACKAGE_DIR=/usr \
    -DCMAKE_TOOLCHAIN_FILE=cmake/aarch64_linux_toolchain.cmake \
    -DEMBEDDED_TARGET=jetson-orin \
    -DCUDA_CTK_VERSION=12.6 \
    -DENABLE_CUTE_DSL=ALL
make -j$(nproc)

Runtime command used:

# N/A — compile-time failure; no runtime reached.
Expected behavior

The build succeeds as promised by the support matrix for the Compatible
Jetson Orin / JetPack 6.2+ / CUDA 12.6 combination. Verified locally: adding
the same compat block used by the sibling runner headers fixes the build:

 #if defined(CUTE_DSL_F16_MOE_ENABLED)

+#include <cuda.h>
+#if defined(TRT_EDGELLM_CUDA_LIBRARY_T_COMPAT)
+#include <cuda_runtime.h>
+#if CUDA_VERSION >= 12000 && CUDA_VERSION < 12080
+typedef CUlibrary cudaLibrary_t;
+static inline cudaError_t cudaLibraryUnload(cudaLibrary_t lib)
+{
+    CUresult r = cuLibraryUnload(lib);
+    return static_cast<cudaError_t>(r);
+}
+#endif // CUDA_VERSION >= 12000 && CUDA_VERSION < 12080
+#endif // TRT_EDGELLM_CUDA_LIBRARY_T_COMPAT
+
 #include "kernels/cuteDslModuleLoader.h"

After the patch, make -j reaches 100% and produces llm_build /
llm_inference / visual_build.

System information (Edge Device)

  • Platform: NVIDIA Jetson AGX Orin Developer Kit
  • Software release: JetPack 6.2
  • CPU architecture: aarch64
  • GPU compute capability: SM87
  • Total device memory: 61Gi (unified)
  • Build type: Release
  • Library versions:
    • TensorRT Edge-LLM version or commit hash: 0.10.0 (release commit 0ce6ce2)
    • CUDA: 12.6 (V12.6.68)
    • TensorRT: 10.3.0.30
    • C++ compiler: GCC 11.4.0
  • CMake options used:
    • CMAKE_TOOLCHAIN_FILE: cmake/aarch64_linux_toolchain.cmake
    • EMBEDDED_TARGET: jetson-orin
    • TRT_PACKAGE_DIR: /usr
  • Any other details that may help:
    • ENABLE_CUTE_DSL=ALL, CUDA_CTK_VERSION=12.6
    • CuTe DSL artifact metadata groups: f16_moe, fmha, gdn, gemm, int4_fp16_gemm, ssd (generated with nvidia-cutlass-dsl[cu12]==4.6.1)
    • Failure is specific to the f16_moe group; builds without it are
      unaffected
    • The same failure applies to x86 CUDA 12 hosts passing
      -DCMAKE_CXX_FLAGS=-DTRT_EDGELLM_CUDA_LIBRARY_T_COMPAT per
      kernelSrcs/int4_fp16_gemm_cutedsl/README.md
      `

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in cpp/kernels/moe/f16_cutedsl/cuteDslF16MoeRunner.h and compare its CUDA compatibility handling with the sibling CuTe DSL runner headers named in the issue. Reproduce with kernelSrcs/build_cutedsl.py and the documented Jetson Orin CMake command; done means the CUDA 12.6 build reaches 100% and produces the listed binaries.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp, python
Domain
build-system, embedded-iot
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.