[Jetson AGX Orin][v0.10.0] Native build may generate XQA kernels for SM80/86/89 instead of SM87

Describe the bug

On Jetson AGX Orin (SM87) with JetPack 6.2 / CUDA 12.6, a native TensorRT Edge-LLM v0.10.0 build can generate XQA cubins for SM80/86/89 instead of SM87.

The Edge-LLM binaries and TensorRT engines can still build successfully, but inference later fails during decoding with:

AttentionPlugin: enqueue failed: No available kernel available for the GQA
Error Code 2: Internal Error (Assertion pluginUtils::isSuccess(status) failed.)
Failed to execute vanilla decoding step for base model.

The problematic CMake configuration reported:

-- XQA Kernels: generating cubins for SM architectures: 80;86;89

although the build host is an aarch64 Jetson AGX Orin (SM87).

The relevant top-level CMakeLists.txt logic is:

if(NOT AARCH64_BUILD)
  set(CMAKE_CUDA_ARCHITECTURES 80;86;89)
  if(CUDA_CTK_VERSION VERSION_GREATER_EQUAL 12.8)
    list(APPEND CMAKE_CUDA_ARCHITECTURES 100a 120)
  endif()
endif()

AARCH64_BUILD is set by cmake/aarch64_linux_toolchain.cmake. In my original native Jetson build, the toolchain file was not specified, so AARCH64_BUILD was not set even though:

$ uname -m
aarch64

cmake/XQACubins.cmake itself already lists SM87 as a supported XQA architecture.

Impact: the failure is difficult to diagnose because compilation and TensorRT engine generation can succeed; the missing SM87 XQA kernel is only exposed later during inference.

Steps/Code to reproduce bug

1. Build TensorRT Edge-LLM v0.10.0 natively on Jetson AGX Orin without specifying cmake/aarch64_linux_toolchain.cmake.

2. Configure Edge-LLM and observe:

-- XQA Kernels: generating cubins for SM architectures: 80;86;89

instead of:

-- XQA Kernels: generating cubins for SM architectures: 87

3. Build the runtime and TensorRT engines.

4. Run inference on an attention configuration requiring the SM87 XQA decode kernel.

5. Runtime decoding fails with:

AttentionPlugin: enqueue failed: No available kernel available for the GQA

Build configuration (problematic native build):

cmake -S . -B build \
  -DCMAKE_BUILD_TYPE=Release \
  -DTRT_PACKAGE_DIR=/usr \
  -DEMBEDDED_TARGET=jetson-orin \
  -DCUDA_CTK_VERSION=12.6 \
  -DCUTE_DSL_ARTIFACT_TAG=sm_87 \
  -DENABLE_CUTE_DSL=ALL

This produced:

-- XQA Kernels: generating cubins for SM architectures: 80;86;89

Working configuration / workaround:

cmake -S . -B build \
  -DCMAKE_TOOLCHAIN_FILE=cmake/aarch64_linux_toolchain.cmake \
  -DCMAKE_BUILD_TYPE=Release \
  -DTRT_PACKAGE_DIR=/usr \
  -DEMBEDDED_TARGET=jetson-orin \
  -DCUDA_CTK_VERSION=12.6 \
  -DCMAKE_CUDA_ARCHITECTURES=87 \
  -DCUTE_DSL_ARTIFACT_TAG=sm_87 \
  -DENABLE_CUTE_DSL=ALL

This correctly produces:

-- XQA Kernels: generating cubins for SM architectures: 87

After rebuilding Edge-LLM and rebuilding the TensorRT engines with this configuration, the same Qwen3-VL inference test succeeds.

Runtime command used:

./build/examples/llm/llm_inference \
  --engineDir /home/orin/vlm-prune/engines/Qwen3-VL-8B-Instruct-INT4-AWQ/llm \
  --multimodalEngineDir /home/orin/vlm-prune/engines/Qwen3-VL-8B-Instruct-INT4-AWQ/visual \
  --inputFile /tmp/qwen3vl_smoke.json \
  --outputFile /tmp/qwen3vl_smoke_output.json

Before rebuilding with SM87 XQA kernels, this fails with:

AttentionPlugin: enqueue failed: No available kernel available for the GQA

After rebuilding with the working configuration:

Successfully captured decoding CUDA graphs for active decoding strategies.
Processing vision inputs
Processing complete: 1/1 batched requests successful

Expected behavior

When building natively on Jetson Orin with:

EMBEDDED_TARGET=jetson-orin

I would expect CMake to either:

1. automatically select the appropriate aarch64 / SM87 configuration, or
2. stop during configuration with a clear message requiring cmake/aarch64_linux_toolchain.cmake.

It may also be useful to validate that CMAKE_CUDA_ARCHITECTURES contains 87 when EMBEDDED_TARGET=jetson-orin.

The current behavior can silently produce a successful build without the required SM87 XQA cubins, with the problem only becoming visible at runtime.

System information (Edge Device)

• Platform (e.g., NVIDIA Jetson Thor): NVIDIA Jetson AGX Orin 64GB
• Software release (e.g., JetPack 7.1): JetPack 6.2
• CPU architecture: aarch64
• GPU compute capability (e.g., SM110 for Jetson Thor): SM87
• Total device memory: 64 GB
• Build type (e.g., Release, Debug): Release
• Library versions:
  • TensorRT Edge-LLM version or commit hash: v0.10.0
  • CUDA: 12.6
  • TensorRT: 10.3
  • C++ compiler (e.g., GCC 11.4): GCC 11.4.0
• CMake options used:
  • CMAKE_TOOLCHAIN_FILE: originally not specified; workaround uses cmake/aarch64_linux_toolchain.cmake
  • EMBEDDED_TARGET: jetson-orin
  • TRT_PACKAGE_DIR: /usr
• Any other details that may help:
  • Qwen3-VL-8B-Instruct was used for end-to-end validation.
  • LLM engine: INT4 AWQ.
  • Vision engine: FP16.
  • Local CUDA 12.6 / SM87 CuTe DSL artifact was used.
  • The SM87 CuTe DSL artifact and f16_moe CUDA 12.6 compatibility issues are separately discussed in #178 and #183.
  • Validated build record: https://github.com/CaesarYP/TensorRT-Edge-LLM/tree/orin-jp62-v0.10.0/build-record

Start with the top-level CMakeLists.txt logic and cmake/aarch64_linux_toolchain.cmake, then inspect cmake/XQACubins.cmake to understand how architectures are selected. Reproduce the native Jetson configuration from the report and compare its output with the toolchain workaround. Done means a native EMBEDDED_TARGET=jetson-orin build selects SM87 or stops with a clear configuration message, and the reported inference test succeeds after rebuilding.