[bug] forwardAsync assertion failed: Unable to get batch slot for reqId
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
Affected versions: https://github.com/NVIDIA/TensorRT-LLM/tree/31ac30e928a2db795799fdcab6be446bfa3a3998 and https://github.com/NVIDIA/TensorRT-LLM/tree/548b5b73106aaf7374955e1c37aad677678ebc7b
[TensorRT-LLM][ERROR] Could not find sequence id in allocated sequence slots
[TensorRT-LLM][ERROR] Encountered an error in forwardAsync function: [TensorRT-LLM][ERROR] Assertion failed: Unable to get batch slot for reqId (/home/jenkins/agent/workspace/LLM/main/L0_MergeRequest/llm/cpp/tensorrt_llm/batch_manager/trtGptModelInflightBatching.cpp:1161)
1 0x411e2b tensorrt_llm::common::throwRuntimeError(char const*, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 78
2 0x7f85d9684401 /opt/distr/wmcore/wmcore_3.4/lib/libtensorrt_llm.so(+0x721401) [0x7f85d9684401]
3 0x7f85db7b7e7c tensorrt_llm::batch_manager::TrtGptModelInflightBatching::forwardAsync(std::__cxx11::list<std::shared_ptr<tensorrt_llm::batch_manager::LlmRequest>, std::allocator<std::shared_ptr<tensorrt_llm::batch_manager::LlmRequest> > > const&) + 1484
4 0x7f85db7e2661 tensorrt_llm::executor::Executor::Impl::forwardAsync(std::__cxx11::list<std::shared_ptr<tensorrt_llm::batch_manager::LlmRequest>, std::allocator<std::shared_ptr<tensorrt_llm::batch_manager::LlmRequest> > >&) + 353
5 0x7f85db7e756f tensorrt_llm::executor::Executor::Impl::executionLoop() + 895
6 0x7f85c59e1a80 /opt/distr/wmcore/wmcore_3.4/lib/libtensorrt_llm_nvrtc_wrapper.so(+0x32c5a80) [0x7f85c59e1a80]
7 0x7f857c9a41ca /opt/distr/wmcore/wmcore_3.4/lib/libpthread.so.0(+0x81ca) [0x7f857c9a41ca]
8 0x7f857bcd08d3 clone + 67
Same error https://github.com/NVIDIA/TensorRT-LLM/issues/1869
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at cpp/tensorrt_llm/batch_manager/trtGptModelInflightBatching.cpp:1161 and trace the forwardAsync path that reports the missing sequence ID and batch slot. Compare the affected revisions with related issue #1869, then reproduce the failure if possible. Done means identifying the cause of the slot lookup failure and adding a regression test or verified fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100