microsoft / microsoft/onnxruntime
[Feature Request] Support for progress and cancellation in compilation APIs
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
#24207 introduced a new API for compiling models. Compilation can potentially take many minutes depending on the model, so it would be useful to augment this API to enable reporting progress (e.g., by some higher-level wrapper or caller), as well as cancellation.
### Describe scenario use case
The scenario would ultimately be to enable finer-grained control over the compilation API by an application (e.g., perhaps as part of its first-run experience the app needs to compile a model that it's bringing with it, so that subsequent inferences are faster; the app may wish to show progress, and also give the user the opportunity to cancel the operation if they so desire). This would in turn enable the developer to produce a nicer user experience.
cc: @adrianlizarraga
Contributor guide
Research direction
Start by reviewing the compilation API introduced in issue #24207 and the surrounding API entry points in ONNX Runtime. Determine how progress reporting and cancellation should be exposed to callers, then clarify the expected behavior and tests for long-running compilation, cancellation, and completion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend-api-design, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100