intel / intel/llm-scaler

Model support request: Qwen3.8-27B on Arc Pro B70

Open
#645 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

## Request

Please add official Intel LLM Scaler support for Qwen3.8-27B on a single Intel Arc Pro B70 32 GB GPU.

## Model

Model family:
- Qwen3.8-27B

Working community checkpoint:
- SergiioB/Qwen3.8-27B-GPTQ-Int4-sym-G128-MTP-BF16

## Current working configuration

Hardware:
- Intel Arc Pro B70 32 GB
- Ubuntu 26.04
- Single GPU

Runtime:
- vLLM XPU
- GPTQ INT4
- FP8 KV cache
- MTP4 speculative decoding
- 128K context
- OpenAI-compatible endpoint
- Tool calling with qwen3_xml parser

Observed performance:
- Approximately 60–70 output tokens/sec on sustained generation
- Concurrent requests require a mixed prefill/speculative-decode patch

## Requested support

Please consider adding:

1. Qwen3.8-27B to the official supported-model matrix
2. An Intel-published Docker image and launch recipe
3. GPTQ INT4 and/or offline FP8 support
4. MTP speculative decoding support
5. Tool-call parser support
6. Validation on a single Arc Pro B70
7. 128K context validation
8. Concurrent-request validation

## Why this matters

Qwen3.8-27B is a strong local coding and agent model and fits on one B70 when quantized. Community recipes already demonstrate that it runs well, but official Intel validation would remove the need for custom patches and pinned nightly images.

Contributor guide

Open the contributing guide

Research direction

Review the supported-model matrix and the existing vLLM XPU configuration, then compare them with the listed Qwen3.8-27B community checkpoint and single Arc Pro B70 setup. Done means defining official support for the requested quantization, speculative decoding, parser, context, concurrency, Docker recipe, and validation coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, ubuntu
Domain
ai-infra-agents, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.