TensorRT-LLM- No free block found.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
System Info
We encountered a runtime crash while performing inference with TensorRT-LLM. The error message indicates a failure in the memory allocation system, specifically that no free memory block was available and the eviction policy failed to recover space.
[TensorRT-LLM][ERROR] Encountered an error in forwardAsync function: [TensorRT-LLM][ERROR] Assertion failed: No free block found. This shouldn't happen! (/src/tensorrt_llm/cpp/tensorrt_llm/batch_manager/evictionPolicy.cpp:89)
Who can help?
No response
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
cannot be 100% reproduced
Expected behavior
efficient memory block management
actual behavior
No
additional notes
no free memory block
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting tensorrt_llm/cpp/tensorrt_llm/batch_manager/evictionPolicy.cpp at line 89 and trace the forwardAsync path that reports the assertion. The issue has no reliable reproduction, so first determine which allocation and eviction conditions lead to the failure. Done means identifying and correcting the no-free-block condition, with a reproducible test or confirmed inference scenario.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100