microsoft / microsoft/onnxruntime-genai
Receiving ROCm execution provider not supported in this build
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
**Describe the bug**
Hi team,
I am currently building OnnxRuntime-GenAI from source for AMD GPUs. With some tweaks to the main branch, I was able to successfully build the `wheel` and install the python package. I was able to use the Model builder to convert LLMs to ROCm supported models. But when I try to run the inference, I am seeing the following error -
```shell
onnxruntime_genai.onnxruntime_genai.OrtException: ROCm execution provider is not enabled in this build.
```
I am sure there is no issue with onnxruntime dependency because I received the following supported EPs from it -
```shell
/workspace# python -c "import onnxruntime as ort; print(ort.get_available_providers())"
['MIGraphXExecutionProvider', 'ROCMExecutionProvider', 'CPUExecutionProvider']
```
Please suggest how I can solve this problem.
**To Reproduce**
Steps to reproduce the behavior:
1. Build and Install OnnxRuntime-Genai from source on a machine with AMD GPU.
**Expected behavior**
I expect no errors and LLMs should output same as they do in Nvidia GPUs.
**Desktop (please complete the following information):**
- OS: RunPod - Ubuntu 22.04 - ROCm 6.1
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the source build on Ubuntu 22.04 with ROCm 6.1 using the steps in the issue, then inspect the build configuration and inference setup for how execution providers are enabled. Confirm that the installed package can use the ROCm execution provider and that the reported inference error no longer occurs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, linux, python, ubuntu
- Domain
- build-system, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100