NVIDIA / NVIDIA/TensorRT-LLM

[Bug]: Compilation error in uballocator when ENABLE_MULTI_DEVICE=0

Open
#6,798 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug Scale-out Testing
Dominant language
Python
Stars
14.7k
Forks
2.8k
Avg merge
2d 23h
Merged PRs (30d)
489

Description

System Info

TRTLLM today's main branch.
cmake 3.27.
gcc 11
RHEL8
CUDA 12.9

Who can help?

@tongyuantongyu

Information
  • The official example scripts
  • My own modified scripts
Tasks
  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)
Reproduction

Try to build without ENABLE_MULTI_DEVICE:

cmake ..  -DCMAKE_VERBOSE_MAKEFILE=ON  -DBUILD_PYBIND=OFF   -DENABLE_MULTI_DEVICE=OFF

See cmake runs correctly.
But compiling fails afterward.

Expected behavior

compilation success.

actual behavior

See compilation error when trying to build the userbuffer kernel:

In file included from TensorRT-LLM/cpp/tensorrt_llm/kernels/userbuffers/ub_interface.h:20,
                 from TensorRT-LLM/cpp/tensorrt_llm/kernels/userbuffers/ub_interface.cpp:16:
TensorRT-LLM/cpp/tensorrt_llm/kernels/userbuffers/ub_allocator.h:38:5: 
error: ‘ncclWindow_t’ does not name a type
   38 |     ncclWindow_t window;
additional notes

It s because when ENABLE_MULTI_DEVICE=OFF, the cmake script does not search for nccl so no src code should try to include/use nccl structs.

Possible solutions:
1 move the UBBuffer inside the ENABLE_MULTI_DEVICE block:

#if ENABLE_MULTI_DEVICE
struct UBBuffer
{
    void* addr;
    int handle;
    size_t size;
    ncclWindow_t window;

2 Keep the UBBuffer class but ifdef the "ncclWindow_t window;" line

struct UBBuffer
{
    void* addr;
    int handle;
    size_t size;
#if ENABLE_MULTI_DEVICE
    ncclWindow_t window;
#endif

I can prepare a PR if you d advice which solution you d prefer.
Best

Before submitting a new issue...
  • Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with cpp/tensorrt_llm/kernels/userbuffers/ub_allocator.h and the include path shown through ub_interface.cpp. Reproduce the build with ENABLE_MULTI_DEVICE=OFF using the reported CMake command, then verify that the userbuffer code no longer requires the unavailable NCCL type and that compilation succeeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp
Domain
build-system
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.