some suggestion
Open
enhancement
- Dominant language
- TypeScript
- Stars
- 979
- Forks
- 132
- Avg merge
- 9h 23m
- Merged PRs (30d)
- 8
Description
ipex-llm seems to already support many quantization methods, but when ipex-llm is used as the back end in the ai playground, no quantization files can be run. It is hoped that ai playground can support quantized files in ipex-llm, and secondly, it is hoped that it can provide the function of opening server like ollama and lm studio, so that ai playground's api can be used in some rag applications. Obviously, inference models like small ones are more suitable for building knowledge base models.
Contributor guide
Assessment
This issue has not been assessed yet.