huggingface / huggingface/optimum-intel

Bug: Qwen3.5-9B INT4 export produces incoherent output (hybrid linear/full attention model)

Open
#1,722 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
620
Forks
270
Avg merge
3d 10h
Merged PRs (30d)
22

Description

## Bug: Qwen3.5-9B INT4 export produces incoherent output (hybrid linear/full attention model)

### System Info

- optimum-intel: 1.27.0.dev0 (from git main, commit d4dd21a)
- openvino: 2026.2.0.dev20260501 (nightly)
- openvino-genai: 2026.2.0.0 (built from source, PR #3801 branch)
- transformers: 5.8.0.dev0 (git main)
- nncf: 2.16.0
- OS: Ubuntu 26.04
- Hardware: Intel Arc Pro B70 (24GB VRAM), 32 CPU cores, 30GB RAM
- compute-runtime: 26.14.37833.4
- IGC: 2.32.7

### Model

Qwen/Qwen3.5-9B — a hybrid linear-attention + full-attention VLM using Gated Delta Rule layers (32 layers in 3:1 linear:full ratio, `linear_conv_kernel_dim=4`, `mamba_ssm_dtype=float32`)

### Export command

```bash
optimum-cli export openvino -m Qwen/Qwen3.5-9B --weight-format int4 --task visual-language-modeling /data/models/qwen35-9b-int4-ov
```

### Description

After exporting Qwen3.5-9B with INT4 quantization, the model produces completely incoherent output on **all devices** (GPU and CPU), using **all inference paths** (openvino-genai ContinuousBatchingPipeline and optimum-intel OVModelForVisualCausalLM).

The same setup with the Qwen3.5-0.8B model exported as FP16 produces correct output ("What is 2+2?" → "4"), confirming that the runtime, drivers, and inference code are all functional.

### Reproduction

```python
import time
import torch
from optimum.intel import OVModelForVisualCausalLM
from transformers import AutoTokenizer

MODEL_DIR = "/data/models/qwen35-9b-int4-ov"

tokenizer = AutoTokenizer.from_pretrained(MODEL_DIR, trust_remote_code=True)
model = OVModelForVisualCausalLM.from_pretrained(MODEL_DIR, device="GPU", trust_remote_code=True)

messages = [{"role": "user", "content": "What is 2+2? Answer with just the number."}]
prompt_text = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, tokenize=False,
extra_context={"enable_thinking": False}
)
inputs = tokenizer(prompt_text, return_tensors="pt")
prompt_tokens = inputs["input_ids"].shape[-1]

# cache_position workaround for transformers 5.8 compatibility
cache_position = torch.arange(prompt_tokens)
output_ids = model.generate(**inputs, max_new_tokens=32, do_sample=False, cache_position=cache_position)
new_ids = output_ids[0][prompt_tokens:]
print(tokenizer.decode(new_ids, skip_special_tokens=True))
```

### Observed output

All three inference paths produce garbage:

| Inference Path | Device | Output |
|---|---|---|
| openvino_genai ContinuousBatchingPipeline | GPU | `"Thinking about 20. # 20. # 20000000000000..."` |
| optimum-intel OVModelForVisualCausalLM | GPU | `"ThinkingThinkingThinkingThinking..."` |
| optimum-intel OVModelForVisualCausalLM | CPU | `"\|,(,,, are,,,,,"` |

### Expected output

`4` (or similar correct response)

### Working comparison

The same code with Qwen3.5-0.8B (FP16, non-MoE) produces correct output:
```
Generated 4 tokens in 0.54s (7.4 tok/s)
Response: 4
```

### Analysis

Since the issue manifests on both CPU and GPU with two completely different inference engines, the problem is in the exported model weights — not the runtime. The INT4 quantization appears to corrupt the novel linear-attention / Gated Delta Rule layers. The 0.8B model works because it is smaller (same architecture but potentially fewer quantization-sensitive parameters at that scale).

Qwen3.5 uses a hybrid architecture with:
- `linear_attention` layers (Gated Delta Rule with conv kernel dim 4)
- `full_attention` layers (standard transformer attention)
- State-space-like components (`mamba_ssm_dtype: float32`)

The linear attention layers have different weight structures from standard transformers (`linear_key_head_dim`, `linear_value_head_dim`, `linear_num_key_heads`, `linear_num_value_heads`) which may not be handled correctly by the INT4 quantization pipeline.

### Questions

1. Have the Gated Delta Rule / linear attention weights been validated for INT4 quantization sensitivity?
2. Should certain layers (e.g., linear attention conv kernels, state projections) be excluded from quantization?
3. Would `--weight-format int8` or `--ratio 0.5` (mixed precision) avoid this issue?
4. Has the 0.8B model been tested with INT4 quantization (to confirm it's a scaling issue vs architectural)?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.