intel / intel/llm-scaler

[Feature Request] Support DFlash2 speculative decoding (vLLM 0.28) for Qwen3.8-27B on dual Arc Pro B60

Open
#656 1 comment 3 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

## Summary

Please add DFlash2 speculative decoding support to `llm-scaler-vllm`, targeting
**Qwen3.8-27B** on a **2x Intel Arc Pro B60** platform.

DFlash2 landed upstream in **vLLM v0.28.0**
("DFlash2 with local convolution and a candidate selector", vllm-project/vllm#52816),
which is ahead of the current llm-scaler base (vLLM 0.21).

## Target configuration

| Item | Value |
|---|---|
| Target model | `Qwen/Qwen3.8-27B` |
| Draft model | `z-lab/Qwen3.8-27B-DFlash2` |
| GPU | Intel Arc Pro B60 x 2 (`--tensor-parallel-size 2`) |
| vLLM base | 0.28.0 (contains DFlash2) |
| Spec decode | `--speculative-config '{"method":"dflash","model":"z-lab/Qwen3.8-27B-DFlash2","num_speculative_tokens":7}'` |

## Performance target

Our acceptance criteria for this platform:

- **Input / output length: 50K / 1K tokens**
- **Output throughput: > 50 tokens/s**

## Environment

- GPU: Intel Arc Pro B60 x 2, Re-sizable BAR enabled
- Current image: `intel/llm-scaler-vllm:0.21.0-b3.1`

## References

- vLLM v0.28.0 release notes (DFlash2)
- vllm-project/vllm#52816 - [Spec Decode] DFlash2: local convolution + candidate selector
- HF: `z-lab/Qwen3.8-27B-DFlash2`

Contributor guide

Open the contributing guide

Research direction

No project files or tests are named. Start by comparing the llm-scaler-vllm vLLM 0.21 base with vLLM 0.28.0 and the referenced DFlash2 change, then reproduce the supplied speculative-config on two Intel Arc Pro B60 GPUs. Done means Qwen/Qwen3.8-27B runs with the DFlash2 draft model for 50K/1K-token workloads and exceeds 50 output tokens per second.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
ai, devops, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.