intel / intel/llm-scaler

Is llm-scaler a replacement for IPEX-LLM?

Open
#283 6 comments 3 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

I saw that Intel archived the updates of the IPEX-LLM project due to security reasons, and now the llm-scaler project has become active. For individual users with GPUs such as B580, Arc 140V/140T and A770, is llm-scaler a replacement for IPEX-LLM? Or is llm-scaler only focused on B60 and multi-GPU usage scenarios, with no support for individual users and single-GPU users?

In addition, what is Intel’s latest solution for individual users running large language models (LLMs)? IPEX-LLM has been discontinued, and the performance of the Vulkan backend or SYCL backend for Llama.cpp and Ollama is not ideal, not as good as that of IPEX-LLM.

Link: https://github.com/intel/ipex-llm

Contributor guide

Open the contributing guide

Research direction

No source file or test is identified; start by reviewing llm-scaler’s current documentation and support information alongside the linked IPEX-LLM project. Done would be a maintainer-confirmed explanation of supported GPUs and single-GPU use, whether llm-scaler replaces IPEX-LLM, and Intel’s current recommendation for individual LLM users.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, ollama
Domain
ai
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.