microsoft / microsoft/onnxruntime
[Feature Request] Model loading progress display or callback?
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
Does onnxruntime.InferenceSession need to add a progress indicator for loading the model? With the development of technology, there are more and more large models, and they often need to wait for a long time when loading. If you add a progress callback or indicator, this It will make the user experience of onnxruntime better.
-----------
onnxruntime.InferenceSession 是否需要添加一个加载模型的进度指示,随着技术的发展,现在有越来越多的大模型,它们在加载时往往需要等待很长的时间,如果添加一个进度回调或指示,这将让onnxruntime的用户体验变的更好。
### Describe scenario use case
In this way, when developing applications and switching models, we can show the user the loading progress of the model, but the interface will still be unresponsive for a few seconds. Although the loading of small models is completed quickly, large models with a G volume will have to wait. With the iteration of technology, future models may be larger. Just like the game loading local resources, a real-time progress value is fed back to facilitate the improvement of application development and operating experience. Thank you~
-----------
这样在开发应用时,在切换模型时,我们可以向用户展示模型的加载进度,而还是几秒的界面无响应。虽然小模型加载很快就完成,但是上G体积的大模型就会等待,随着技术的迭代,以后的模型也许体积更大。就像游戏加载本地资源一样,反馈一个实时进度值,以方便改善应用的开发与操作体验。谢谢~
Contributor guide
Research direction
Start with the onnxruntime.InferenceSession model-loading entry point and determine where progress could be measured for large models. Define whether the feature should expose a callback, a progress indicator, or both, including behavior when progress cannot be reported; done means applications can observe loading progress without changing inference behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- api, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100