microsoft / microsoft/onnxruntime-genai

Configuring SessionOptions programmatically

Open
#1,393 5 comments 0 reactions 0 assignees View on GitHub
ep:DML
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

I needed to enable verbose logging to troubleshoot an issue I was having. As far as I know, this is done by configuring the `LogSeverityLevel` property of the `SessionOptions`.

I'm currently using **Microsoft.Extensions.AI** and the `OnnxRuntimeGenAIChatClient`. The only mechanism to configure the execution environment that appears to be currently available is to create a `Config` object and use it to construct a `Model`, which is then used to initialize the client. However, even though the `Config` class has a `SetProviderOption` method, it seems to be missing a `SetOption` method.

For example:

``` C#
...
using Config config = new(modelPath);

// THE FOLLOWING IS NOT CURRENTLY POSSIBLE
config.SetOption("log_severity_level", OrtLoggingLevel.ORT_LOGGING_LEVEL_VERBOSE);

using Model model = new(config);
using var client = new OnnxRuntimeGenAIChatClient(model, true, options);
...
```

I'm aware that you can do this by setting the property in `genai_config.json`, but there seems to be missing a mechanism to do this programmatically.
``` JSON
"session_options": {
"log_id": "onnxruntime-genai",
"log_severity_level": 0,
"provider_options": [
{
"dml": {}
}
]
}
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the Config API, especially SetProviderOption, and how the session_options section of genai_config.json is consumed. Define the programmatic equivalent for configuring log_severity_level and verify that it can be used when constructing a Model for OnnxRuntimeGenAIChatClient.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, csharp
Domain
api
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.