microsoft / microsoft/onnxruntime-genai
Configuring SessionOptions programmatically
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
I needed to enable verbose logging to troubleshoot an issue I was having. As far as I know, this is done by configuring the `LogSeverityLevel` property of the `SessionOptions`.
I'm currently using **Microsoft.Extensions.AI** and the `OnnxRuntimeGenAIChatClient`. The only mechanism to configure the execution environment that appears to be currently available is to create a `Config` object and use it to construct a `Model`, which is then used to initialize the client. However, even though the `Config` class has a `SetProviderOption` method, it seems to be missing a `SetOption` method.
For example:
``` C#
...
using Config config = new(modelPath);
// THE FOLLOWING IS NOT CURRENTLY POSSIBLE
config.SetOption("log_severity_level", OrtLoggingLevel.ORT_LOGGING_LEVEL_VERBOSE);
using Model model = new(config);
using var client = new OnnxRuntimeGenAIChatClient(model, true, options);
...
```
I'm aware that you can do this by setting the property in `genai_config.json`, but there seems to be missing a mechanism to do this programmatically.
``` JSON
"session_options": {
"log_id": "onnxruntime-genai",
"log_severity_level": 0,
"provider_options": [
{
"dml": {}
}
]
}
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the Config API, especially SetProviderOption, and how the session_options section of genai_config.json is consumed. Define the programmatic equivalent for configuring log_severity_level and verify that it can be used when constructing a Model for OnnxRuntimeGenAIChatClient.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, csharp
- Domain
- api
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100