microsoft / microsoft/onnxruntime-genai

Building from Source for Jetson

Open
#818 3 comments 0 reactions 0 assignees View on GitHub
ep:CUDA platform:jetson
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

Hi,

I'm trying to build onnxruntime-genai with cuda support on Jetson device. I have onnxruntime-1.18.0 and onnxruntime-gpu 1.18.0.

I have placed the .so files in ort/lib/ and .h files in ort/include as instructed.

But when building, it fetches onnxruntime version 1.17.1 into the _deps. Does this version have to match the onnxruntime I have installed on the machine ? I'm not able to use CUDA to run genai.

Kindly help me on how to change the onnxruntime source version getting fetched by cmake to customizable option.

Thanks in advance.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by examining the CMake configuration that fetches onnxruntime 1.17.1 into _deps, along with the ort/lib and ort/include layout described in the issue. Determine whether the source version can be selected through a build option or whether the local 1.18.0 files are ignored. Done means the build can use a customizable onnxruntime version and the Jetson CUDA build behavior is documented or verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp
Domain
build-system
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.