Model support request: Qwen3.8-27B on Arc Pro B70
- Dominant language
- C++
- Stars
- 529
- Forks
- 80
- Avg merge
- 9h 7m
- Merged PRs (30d)
- 38
Description
## Request
Please add official Intel LLM Scaler support for Qwen3.8-27B on a single Intel Arc Pro B70 32 GB GPU.
## Model
Model family:
- Qwen3.8-27B
Working community checkpoint:
- SergiioB/Qwen3.8-27B-GPTQ-Int4-sym-G128-MTP-BF16
## Current working configuration
Hardware:
- Intel Arc Pro B70 32 GB
- Ubuntu 26.04
- Single GPU
Runtime:
- vLLM XPU
- GPTQ INT4
- FP8 KV cache
- MTP4 speculative decoding
- 128K context
- OpenAI-compatible endpoint
- Tool calling with qwen3_xml parser
Observed performance:
- Approximately 60–70 output tokens/sec on sustained generation
- Concurrent requests require a mixed prefill/speculative-decode patch
## Requested support
Please consider adding:
1. Qwen3.8-27B to the official supported-model matrix
2. An Intel-published Docker image and launch recipe
3. GPTQ INT4 and/or offline FP8 support
4. MTP speculative decoding support
5. Tool-call parser support
6. Validation on a single Arc Pro B70
7. 128K context validation
8. Concurrent-request validation
## Why this matters
Qwen3.8-27B is a strong local coding and agent model and fits on one B70 when quantized. Community recipes already demonstrate that it runs well, but official Intel validation would remove the need for custom patches and pinned nightly images.
Contributor guide
Research direction
Review the supported-model matrix and the existing vLLM XPU configuration, then compare them with the listed Qwen3.8-27B community checkpoint and single Arc Pro B70 setup. Done means defining official support for the requested quantization, speculative decoding, parser, context, concurrency, Docker recipe, and validation coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, ubuntu
- Domain
- ai-infra-agents, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100