microsoft / microsoft/onnxruntime
[Feature Request] Get GPT4All to work with onnxruntime
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
I would like to run this model with onnxruntime:
[GPT4all ](https://huggingface.co/Sosaka/GPT4All-7B-4bit-ggml)
I don't believe you can currently successfully convert this into an onnx runtime and run it with onnxruntime.
If I am wrong please describe the process. Thanks.
If not, can you suggest an alternative framework other than onnxruntime to run this with c#/DirectML?
### Describe scenario use case
To run this model and similar models.
Contributor guide
Research direction
Start by investigating whether the linked GPT4All-7B-4bit-ggml model can be converted and run with ONNX Runtime. Review the repository’s existing C#/DirectML support and model-conversion guidance, if available. Done means a documented working path, or a confirmed limitation with an alternative framework.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- ai, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100