microsoft / microsoft/onnxruntime

[Feature Request] Support for progress and cancellation in compilation APIs

Open
#24,573 0 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

#24207 introduced a new API for compiling models. Compilation can potentially take many minutes depending on the model, so it would be useful to augment this API to enable reporting progress (e.g., by some higher-level wrapper or caller), as well as cancellation.

### Describe scenario use case

The scenario would ultimately be to enable finer-grained control over the compilation API by an application (e.g., perhaps as part of its first-run experience the app needs to compile a model that it's bringing with it, so that subsequent inferences are faster; the app may wish to show progress, and also give the user the opportunity to cancel the operation if they so desire). This would in turn enable the developer to produce a nicer user experience.

cc: @adrianlizarraga

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the compilation API introduced in issue #24207 and the surrounding API entry points in ONNX Runtime. Determine how progress reporting and cancellation should be exposed to callers, then clarify the expected behavior and tests for long-running compilation, cancellation, and completion.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.