intel/llm-scaler

View on GitHub

Stars: 529
Forks: 80
Open beginner issues: 0
Indexed issues: 115
Avg merge: 9h 7m
Merged PRs (30d): 38

Dominant language: C++
License: Apache-2.0
Last GitHub push: Sep 16, 2026
Latest indexed: Sep 17, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed

Open All

115 open issues indexed

InferenceMAX Open

intel/llm-scaler#115 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Do we support whisper-large-v3-turbo, bge-m3, CosyVoice2? Open

omni user issue

intel/llm-scaler#169 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Will we ever get training tools like axolotl for gpt-oss type models. Because 8 B60's RIP through distill Open

enhancement user issue

intel/llm-scaler#181 · 6 comments · 1 reaction · 1 assignee · Indexed Sep 17, 2026
FSDP Fails with FP8 FLUX.2 Model Open

omni

intel/llm-scaler#201 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Enabling Upstage's Solar-Open-100B Open

vllm

intel/llm-scaler#247 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Installing on Ubuntu 25.04 Server fails Open

platform user issue

intel/llm-scaler#268 · 5 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Is llm-scaler a replacement for IPEX-LLM? Open

intel/llm-scaler#283 · 6 comments · 3 reactions · 0 assignees · Indexed Sep 17, 2026
vllm fp8 quantization fails for Llama-3.1-8B-Instruct Open

vllm

intel/llm-scaler#287 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
minicpm-v-4_5 vllm fail with larger batch size (>=16) Open

intel/llm-scaler#304 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] intel/llm-scaler-vllm:latest (1.4 / vLLM 0.14.0 / PyTorch 2.10): XPU OutOfMemoryError on first tensor allocation when 2x Arc B580 are visible Open

intel/llm-scaler#305 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Differences between intel/vllm (latest) and intel/llm-scaler (latest) Open

vllm

intel/llm-scaler#309 · 8 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Qwen3-32B vllm speculative decoding benchmark failed Open

vllm

intel/llm-scaler#312 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Could not run Qwen/Qwen3.5-27B-GPTQ-Int4 with release 0.14.0-b8.1 Open

vllm

intel/llm-scaler#324 · 5 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
torch._dynamo.exc.Unsupported: _has_xmx crash when running 32B model on single Arc B60 Pro Open

vllm

intel/llm-scaler#325 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
IndexTTS2 is running on CPU instead of GPU in omni Open

omni

intel/llm-scaler#326 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
SeedVR2 torch.compile() can't run with llm-scaler-omni:0.1.0-b6 on B70 Open

omni

intel/llm-scaler#333 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Clarify vllm commitment plan that was announced Open

intel/llm-scaler#336 · 10 comments · 5 reactions · 0 assignees · Indexed Sep 17, 2026
[Bug] GLM-4.7-Flash MTP fails with sample_recovered_tokens_kernel build error on Intel XPU (ZE_RESULT_ERROR_MODULE_BUILD_FAILURE) Open

intel/llm-scaler#344 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Using 2X B70's is throwing UR_RESULT_ERROR_UNKNOWN error with VLLM Open

intel/llm-scaler#347 · 9 comments · 2 reactions · 3 assignees · Indexed Sep 17, 2026
Determining which model has best compatability with llm-scaler Open

intel/llm-scaler#348 · 12 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
There is no file_parse endpoint for MinerU2.5-2509-1.2B model Open

intel/llm-scaler#349 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Intel Arc Pro B50 support question ? Open

intel/llm-scaler#350 · 8 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Inference Qwen3-32B BUG for llm-scaler-vllm:0.14.0-b8.1 Open

vllm

intel/llm-scaler#355 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
LTX-2.3 can't run with llm-scaler-omni:0.1.0-b6 Open

intel/llm-scaler#358 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
GLM-4.7-Flash instance is crashed Open

intel/llm-scaler#364 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
How about posting launchers for "validated models?" like Qwen3.5 Open

intel/llm-scaler#366 · 5 comments · 1 reaction · 1 assignee · Indexed Sep 17, 2026
Missing Qwen3.5MoeForConditionalGeneration support Open

intel/llm-scaler#367 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Need to set up proper permissions for `/dev/dri/{card,render}*` Open

intel/llm-scaler#370 · 5 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Very low performance with Qwen3.5-27b-GPTQ-Int4 Open

intel/llm-scaler#371 · 15 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Hardware Support: B580 compatibility and VRAM pooling with B70 via USB4 eGPU Open

intel/llm-scaler#378 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
dflash Open

intel/llm-scaler#379 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
MiniCPM-V 4.5 regression on Intel LLM Scaler `0.14.0-b8.2` Open

intel/llm-scaler#380 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Request for Ubuntu 26.04 Support in Bare Metal Environment Open

intel/llm-scaler#381 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Unable to run Qwen3.6-35B-A3B FP8 on 2x B70s using `llm-scaler-vllm:0.14.0-b8.2` Open

intel/llm-scaler#382 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] torch.compile fails on FP8 + XPU: `torch_ipex.fp8_gemm_w8a16` lacks fake-tensor implementation, forcing --enforce-eager (~10–20% perf cost) Open

intel/llm-scaler#383 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
[Bug] MTP speculative_config fails to run Crash when enabling MTP speculative decoding Open

intel/llm-scaler#386 · 0 comments · 1 reaction · 1 assignee · Indexed Sep 17, 2026
V1 engine breaks block size for gpt-oss-120b Open

intel/llm-scaler#390 · 1 comment · 1 reaction · 1 assignee · Indexed Sep 17, 2026
VLLM seems be hang Open

intel/llm-scaler#391 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Arc Pro B50 (BMG-G21) — torch.compile produces wrong output / SYCL JIT crash on vLLM Open

intel/llm-scaler#392 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Running 4 * B70 met warning on conrainer “intel/llm-scaler-vllm:0.14.0-b8.1” and got low perf Open

intel/llm-scaler#397 · 4 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
AWQ quantized models fail with zeMemOpenIpcHandle ZE_RESULT_ERROR_INVALID_ARGUMENT on dual Arc Pro B70 (Linux + llm-scaler vLLM, tp=2) Open

intel/llm-scaler#398 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
AWQ quantization models have abnormal memory usage / OOM on PTL 358H Open

intel/llm-scaler#402 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
docker 环境怎么配的 Open

intel/llm-scaler#408 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Qwen3.5 27b int4 autoaround Tool Calling Benchmark Fails on Intel Arc Pro B70 ×2 with vLLM 0.14.0-b8.2.1 Open

intel/llm-scaler#412 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Docker image intel/llm-scaler-vllm:0.14.0-b8.2.1 does not support Qwen3.5-35B-A3B-GPTQ-Int4 and Qwen3.5-35B-A3B-FP8 Open

intel/llm-scaler#415 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Qwen3.6 35B-A3B failed to start in intel/llm-scaler-vllm:0.14.0-b8.2.1 with downgrading transformer Open

intel/llm-scaler#423 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
intel/llm-scaler-vllm:1.4 — TP=2 fails with UR_RESULT_ERROR_DEVICE_LOST during determine_available_memory on dual Arc B50 Pro (regression vs 1.3) Open

intel/llm-scaler#424 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Intel Arc B70 / BMG G31: IPEX-Ollama oneAPI runner crashes on qwen2.5:0.5b, CPU path works Open

intel/llm-scaler#426 · 5 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Intel scaler omni upgrade Open

intel/llm-scaler#428 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Qwen3_5ForConditionalGeneration fails to load on single Intel Arc Pro B70 (32GB) due to SYCL crash in vllm registry subprocess Open

intel/llm-scaler#435 · 10 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026

intel/llm-scaler

Get new issues in your inbox