microsoft / microsoft/onnxruntime
[Feature Request] Add ability to set memory pool limit for tactic search in TensorRT EP
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
Engine creation by TensorRT uses the internal flag nvinfer1::MemoryPoolType::kTACTIC_DRAM to set a limit for tactic search, without this memory during tactic selection (i.e. engine creation) is not set and significantly higher spikes of memory is observed than set by workspace limit (nvinfer1::MemoryPoolType::kWORKSPACE)
I want to add the ability to set this variable in the TensorRT EP via a EP session flag.
Contributor guide
Research direction
Start with TensorRT EP session-flag handling and the engine-creation path where MemoryPoolType::kTACTIC_DRAM and kWORKSPACE are used. Define completion as allowing the tactic-search memory limit to be set independently through an EP session flag, then verify that the configured limit affects engine creation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100