microsoft / microsoft/onnxruntime-genai
Cuda / DirectML question
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
Hi there, you made fantastic framework for llms. But what I find very confusing is how to run this on cuda and direct ml. I simply don't know how to do it in C#..
I there any example? Second question is, do I have to provide different model per cuda, cpu and directml or can it run seamlessly? Or is there a way to convert model to support all or combination of providers? as far as I know onnx it self provides seamles support that's why it's a bit confusing.
My use case is to deploy to user's device a model and based on his capabilities to choose the provider which can provide best performance. But not in the opposite direction, because I expect my user to know nothing about the ML it self.
Thank you ✌️
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no repository files, tests, or entry points. First determine whether C# examples and provider-selection guidance belong in the repository, then document the supported CUDA, CPU, and DirectML workflow and verify the examples against the relevant runtime setup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, csharp
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100