microsoft / microsoft/onnxruntime
Enable clients to supply their own threads when inferencing on CPU
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 8h
- Merged PRs (30d)
- 179
Description
Is your feature request related to a problem? Please describe.
We have a system with an existing thread pool infrastructure. We run with tight resource & performance constraints and would like to have complete control of the threading in our program. It would be helpful if one could somehow subscribe his / her own threadpool with the ONNXRuntime environment / session.
System information
- ONNX Runtime version (you are using): 1.71, Windows 10
Describe the solution you'd like
An interface / API that enables onnxruntime to post tasks to a user provided thread pool.
Describe alternatives you've considered
Currently we must use the ONNXRuntime threadpool, which is wasteful as we already have a thread pool in place.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files or tests; start by reviewing the ONNX Runtime environment and session threading interfaces and the CPU inference path. Done means defining an API that lets clients provide an existing thread pool and clarifying how it interacts with ONNX Runtime's built-in pool.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100