Does setting setTensorAddress on every inference call impact throughput?
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Hello,
I have been using the older execute inference API in TensorRT for several years. Due to a recent GPU upgrade, I am now migrating to a newer version of TensorRT.
I noticed that when using the enqueueV3 or enqueueV2 APIs, the tensor address must be specified for each inference call via setTensorAddress.
Could you clarify if calling setTensorAddress on every inference introduces any overhead that might impact overall performance or throughput?
Thank you.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the enqueueV2 and enqueueV3 entry points and the setTensorAddress API mentioned in the issue. Determine whether setting tensor addresses on each inference call adds measurable overhead or affects throughput, then provide a documented answer; no source file or test is identified in the payload.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning, performance
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100