microsoft / microsoft/onnxruntime-genai
[Linux] [C#] [0.5.2] Publish/Build Binaries with Microsoft.ML.OnnxRuntimeGenAI.Cuda on Linux containing unnecessary large DLL files
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
**Describe the bug**
When I build or publish linux-x64 binary with `Microsoft.ML.OnnxRuntimeGenAI.Cuda 0.5.2`, the final output folder will contain many unnecessary DLLs (`onnxruntime*.dll`), **especially `onnxruntime_providers_cuda.dll` which is very large (>600MB)**

Could you please fix to stop publishing these `onnxruntime*.dll` files for linux-x64 when `dotnet build/dotnet publish`, otherwise the final binaries size will be double (both cuda.so and cuda.sll >600MB)
**To Reproduce**
Steps to reproduce the behavior:
- test.csproj
```
Exe
net8
enable
enable
true
false
true
```
- Program.cs
```
namespace MyTest;
using Microsoft.ML.OnnxRuntime;
public class Program
{
public static async Task Main(string[] args)
{
const string modelPath = "/home/jiaxuwu/.aitk/models/onnx/cuda/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32.onnx";
using var session = new InferenceSession(modelPath);
}
}
```
- `dotnet publish test.csproj --runtime linux-x64`
- `ll -h bin/Release/net8/linux-x64/publish/onnxruntime*.dll`
- `ll -h bin/Release/net8/linux-x64/onnxruntime*.dll`
**Expected behavior**
`Microsoft.ML.OnnxRuntimeGenAI.Cuda` shouldn't publish any unnecessary very large files on Linux
**Screenshots**

**Desktop (please complete the following information):**
- OS: WLS(Windows Subsystem for Linux)
```
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.5 LTS
Release: 22.04
Codename: jammy
```
- .NET
```
$ dotnet --list-sdks
8.0.110 [/usr/lib/dotnet/sdk]
```
- Nuget Package Version: `Microsoft.ML.OnnxRuntimeGenAI.Cuda 0.5.2`
**Additional context**
**Another package `Microsoft.ML.OnnxRuntimeGenAI 0.5.2` has the similar issue on MacOS**
MacOS final output binaries will also contain several unnecessary DLL files with `Microsoft.ML.OnnxRuntimeGenAI 0.5.2`
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the provided test.csproj and Program.cs, then run dotnet publish test.csproj --runtime linux-x64 and inspect the generated onnxruntime*.dll files in the publish and build directories. The fix is complete when the Linux output no longer includes the unnecessary large DLL files while retaining the required CUDA binaries.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, linux, ubuntu
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100