microsoft / microsoft/onnxruntime

[Feature Request] Get GPT4All to work with onnxruntime

Open
#15,290 1 comment 5 reactions 0 assignees View on GitHub
feature request model:transformer
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

I would like to run this model with onnxruntime:

[GPT4all ](https://huggingface.co/Sosaka/GPT4All-7B-4bit-ggml)

I don't believe you can currently successfully convert this into an onnx runtime and run it with onnxruntime.

If I am wrong please describe the process. Thanks.

If not, can you suggest an alternative framework other than onnxruntime to run this with c#/DirectML?

### Describe scenario use case

To run this model and similar models.

Contributor guide

Open the contributing guide

Research direction

Start by investigating whether the linked GPT4All-7B-4bit-ggml model can be converted and run with ONNX Runtime. Review the repository’s existing C#/DirectML support and model-conversion guidance, if available. Done means a documented working path, or a confirmed limitation with an alternative framework.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
ai, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.