Very low performance with Qwen3.5-27b-GPTQ-Int4

model: https://huggingface.co/Qwen/Qwen3.5-27B-GPTQ-Int4
GPU: B70 x2
ubuntu 25.04
GPU "driver": multi-arc-bmg-offline-installer-26.5.6.1-dual

```
docker run -td \
--privileged \
--net=host \
--device=/dev/dri \
--name=lsv-container-qwen35 \
-v /home/edison/LLM:/llm/models/ \
-e no_proxy=localhost,127.0.0.1 \
-e http_proxy=$http_proxy \
-e https_proxy=$https_proxy \
--shm-size="32g" \
--entrypoint /bin/bash \
intel/llm-scaler-vllm:0.14.0-b8.1

```
```
vllm serve \
--model /llm/models/Qwen3.5-27b-GPTQ-Int4 \
--served-model-name Qwen3.5-27b-GPTQ-Int4 \
--enforce-eager \
--port 8000 \
--host 0.0.0.0 \
--trust-remote-code \
--disable-sliding-window \
--gpu-memory-util=0.70 \
--block-size 32 \
--max-model-len=16384 \
--max-num-batched-tokens=8192 \
--reasoning-parser qwen3 \
--language-model-only \
--allow-deprecated-quantization \
-tp=2

```

```
llama-benchy --base-url "http://0.0.0.0:8000/v1" --model Qwen3.5-27b-GPTQ-Int4 --tokenizer "/home/edison/LLM/Qwen3.5-27b-GPTQ-Int4" --pp 512 2048 8192 --tg 32 128 --runs 5

| model | test | t/s | peak t/s | ttfr (ms) | est_ppt (ms) | e2e_ttft (ms) |
|:----------------------|-------:|-----------------:|-------------:|----------------:|----------------:|----------------:|
| Qwen3.5-27b-GPTQ-Int4 | pp512 | 2310.58 ± 309.64 | | 228.34 ± 38.13 | 227.05 ± 38.13 | 228.44 ± 38.12 |
| Qwen3.5-27b-GPTQ-Int4 | tg32 | 11.56 ± 0.07 | 12.00 ± 0.00 | | | |
| Qwen3.5-27b-GPTQ-Int4 | pp512 | 2481.01 ± 47.78 | | 208.06 ± 4.04 | 206.77 ± 4.04 | 208.17 ± 4.03 |
| Qwen3.5-27b-GPTQ-Int4 | tg128 | 11.68 ± 0.07 | 12.00 ± 0.00 | | | |
| Qwen3.5-27b-GPTQ-Int4 | pp2048 | 2680.95 ± 110.79 | | 766.97 ± 33.73 | 765.67 ± 33.73 | 767.07 ± 33.74 |
| Qwen3.5-27b-GPTQ-Int4 | tg32 | 11.74 ± 0.07 | 12.00 ± 0.00 | | | |
| Qwen3.5-27b-GPTQ-Int4 | pp2048 | 2734.92 ± 4.99 | | 750.43 ± 1.23 | 749.13 ± 1.23 | 750.51 ± 1.26 |
| Qwen3.5-27b-GPTQ-Int4 | tg128 | 11.83 ± 0.02 | 12.00 ± 0.00 | | | |
| Qwen3.5-27b-GPTQ-Int4 | pp8192 | 2695.04 ± 34.16 | | 3041.81 ± 38.02 | 3040.51 ± 38.02 | 3041.88 ± 38.01 |
| Qwen3.5-27b-GPTQ-Int4 | tg32 | 11.74 ± 0.03 | 12.00 ± 0.00 | | | |
| Qwen3.5-27b-GPTQ-Int4 | pp8192 | 2682.34 ± 1.88 | | 3055.72 ± 2.14 | 3054.42 ± 2.14 | 3055.83 ± 2.15 |
| Qwen3.5-27b-GPTQ-Int4 | tg128 | 11.80 ± 0.04 | 12.00 ± 0.00 | | | |

```