NVIDIA / NVIDIA/TensorRT-LLM

[bug] forwardAsync assertion failed: Unable to get batch slot for reqId

Open
#2,626 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
14.7k
Forks
2.8k
Avg merge
2d 23h
Merged PRs (30d)
489

Description

Affected versions: https://github.com/NVIDIA/TensorRT-LLM/tree/31ac30e928a2db795799fdcab6be446bfa3a3998 and https://github.com/NVIDIA/TensorRT-LLM/tree/548b5b73106aaf7374955e1c37aad677678ebc7b

[TensorRT-LLM][ERROR] Could not find sequence id in allocated sequence slots
[TensorRT-LLM][ERROR] Encountered an error in forwardAsync function: [TensorRT-LLM][ERROR] Assertion failed: Unable to get batch slot for reqId (/home/jenkins/agent/workspace/LLM/main/L0_MergeRequest/llm/cpp/tensorrt_llm/batch_manager/trtGptModelInflightBatching.cpp:1161)
1             0x411e2b tensorrt_llm::common::throwRuntimeError(char const*, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 78
2       0x7f85d9684401 /opt/distr/wmcore/wmcore_3.4/lib/libtensorrt_llm.so(+0x721401) [0x7f85d9684401]
3       0x7f85db7b7e7c tensorrt_llm::batch_manager::TrtGptModelInflightBatching::forwardAsync(std::__cxx11::list<std::shared_ptr<tensorrt_llm::batch_manager::LlmRequest>, std::allocator<std::shared_ptr<tensorrt_llm::batch_manager::LlmRequest> > > const&) + 1484
4       0x7f85db7e2661 tensorrt_llm::executor::Executor::Impl::forwardAsync(std::__cxx11::list<std::shared_ptr<tensorrt_llm::batch_manager::LlmRequest>, std::allocator<std::shared_ptr<tensorrt_llm::batch_manager::LlmRequest> > >&) + 353
5       0x7f85db7e756f tensorrt_llm::executor::Executor::Impl::executionLoop() + 895
6       0x7f85c59e1a80 /opt/distr/wmcore/wmcore_3.4/lib/libtensorrt_llm_nvrtc_wrapper.so(+0x32c5a80) [0x7f85c59e1a80]
7       0x7f857c9a41ca /opt/distr/wmcore/wmcore_3.4/lib/libpthread.so.0(+0x81ca) [0x7f857c9a41ca]
8       0x7f857bcd08d3 clone + 67

Same error https://github.com/NVIDIA/TensorRT-LLM/issues/1869

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at cpp/tensorrt_llm/batch_manager/trtGptModelInflightBatching.cpp:1161 and trace the forwardAsync path that reports the missing sequence ID and batch slot. Compare the affected revisions with related issue #1869, then reproduce the failure if possible. Done means identifying the cause of the slot lookup failure and adding a regression test or verified fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.