microsoft / microsoft/onnxruntime-genai
OnnxRuntimeGenAIChatClientOptions support DeepSeek-r1-Distill-14b
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
I am not sure if the OnnxRuntimeGenAIChatClientOptions of the following deepseek model is correct. The value of message.Role.Value is fixed to "user". The user prompt tag of this deepseek model is < | User | >.
model:https://huggingface.co/onnxruntime/DeepSeek-R1-Distill-ONNX/tree/main/deepseek-r1-distill-qwen-14B/gpu/gpu-int4-rtn-block-32
```
public static OnnxRuntimeGenAIChatClientOptions GetPhi4() => new()
{
StopSequences = ["<|system|>", "<|user|>", "<|assistant|>", "<|end|>"],
PromptFormatter = static (messages, _) =>
{
var prompt = new StringBuilder();
foreach (var message in messages)
foreach (var content in message.Contents.OfType())
prompt.Append($"<|{message.Role.Value}|>\n{content.Text}<|end|>\n");
return prompt.Append("<|assistant|>\n").ToString();
}
};
public static OnnxRuntimeGenAIChatClientOptions GetDeepSeek() => new()
{
StopSequences = ["<|end▁of▁sentence|>"],
PromptFormatter = static (messages, _) =>
{
var prompt = new StringBuilder();
foreach (var message in messages)
foreach (var content in message.Contents.OfType())
prompt.Append($"<|begin▁of▁sentence|><|{message.Role.Value}|>\n{content.Text}<|end▁of▁sentence|>\n");
return prompt.Append("<|Assistant|>\n").ToString();
}
};
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the OnnxRuntimeGenAIChatClientOptions entry point and the GetDeepSeek formatter shown in the issue. Compare its role and prompt tokens with the linked DeepSeek-R1-Distill model format. Done means the options produce the model's expected user and assistant tags and stop sequence.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100