microsoft / microsoft/onnxruntime-genai

[Linux] [C#] [0.5.2] Publish/Build Binaries with Microsoft.ML.OnnxRuntimeGenAI.Cuda on Linux containing unnecessary large DLL files

Open
#1,135 6 comments 0 reactions 0 assignees View on GitHub
ep:CUDA
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

**Describe the bug**
When I build or publish linux-x64 binary with `Microsoft.ML.OnnxRuntimeGenAI.Cuda 0.5.2`, the final output folder will contain many unnecessary DLLs (`onnxruntime*.dll`), **especially `onnxruntime_providers_cuda.dll` which is very large (>600MB)**
![Image](https://github.com/user-attachments/assets/802e5c20-98b7-4d52-87af-fca012c1bb36)

Could you please fix to stop publishing these `onnxruntime*.dll` files for linux-x64 when `dotnet build/dotnet publish`, otherwise the final binaries size will be double (both cuda.so and cuda.sll >600MB)

**To Reproduce**
Steps to reproduce the behavior:
- test.csproj
```


Exe
net8
enable
enable
true
false
true





```
- Program.cs
```
namespace MyTest;

using Microsoft.ML.OnnxRuntime;

public class Program
{
public static async Task Main(string[] args)
{
const string modelPath = "/home/jiaxuwu/.aitk/models/onnx/cuda/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32.onnx";
using var session = new InferenceSession(modelPath);
}
}

```
- `dotnet publish test.csproj --runtime linux-x64`
- `ll -h bin/Release/net8/linux-x64/publish/onnxruntime*.dll`
- `ll -h bin/Release/net8/linux-x64/onnxruntime*.dll`

**Expected behavior**
`Microsoft.ML.OnnxRuntimeGenAI.Cuda` shouldn't publish any unnecessary very large files on Linux

**Screenshots**
![Image](https://github.com/user-attachments/assets/802e5c20-98b7-4d52-87af-fca012c1bb36)

**Desktop (please complete the following information):**
- OS: WLS(Windows Subsystem for Linux)
```
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.5 LTS
Release: 22.04
Codename: jammy
```
- .NET
```
$ dotnet --list-sdks
8.0.110 [/usr/lib/dotnet/sdk]
```
- Nuget Package Version: `Microsoft.ML.OnnxRuntimeGenAI.Cuda 0.5.2`

**Additional context**
**Another package `Microsoft.ML.OnnxRuntimeGenAI 0.5.2` has the similar issue on MacOS**
MacOS final output binaries will also contain several unnecessary DLL files with `Microsoft.ML.OnnxRuntimeGenAI 0.5.2`

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the provided test.csproj and Program.cs, then run dotnet publish test.csproj --runtime linux-x64 and inspect the generated onnxruntime*.dll files in the publish and build directories. The fix is complete when the Linux output no longer includes the unnecessary large DLL files while retaining the required CUDA binaries.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, linux, ubuntu
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.