GLM-5.3-Flash Q2 performance on AMD Strix Halo / Radeon 8060S

# GLM-5.3-Flash Q2 performance on AMD Strix Halo / Radeon 8060S

## Hardware

- **SoC:** AMD Ryzen AI Max+ 395
- **GPU:** AMD Radeon 8060S Graphics (`gfx1151`, reported by DS4 as `sm_115`)
- **Unified memory:** 128 GiB
- **OS:** Arch Linux
- **ROCm:** 7.2
- **Mesa/Vulkan:** 26.3
- **Note:** DS4 tests use **ROCm**, not Vulkan. Vulkan 26.3 is used only for the llama.cpp comparison below.

---

## DS4 build

Repository:

```bash
git clone https://github.com/antirez/ds4.git
cd ds4
git checkout glm-5.3-flash
```

Build:

```bash
make clean
make strix-halo -j"$(nproc)"
```

ROCm backend successfully initialized:

```text
ds4: ROCm backend initialized on AMD Radeon 8060S Graphics (sm_115)
```

---

# Test 1 — antirez GLM-5.3-Flash Q2 / DS4 / ROCm

## Model

Antirez GLM-5.3-Flash Q2:

```text
GLM-5.3-Flash-Q2.gguf
```

DS4 reports the full GGUF size as approximately:

```text
89.88 GiB
```

---

## Test 1A — 500K context / 32 GiB expert cache

Command:

```bash
./ds4 \
-m "$HOME/models/GLM-5.3-Flash-Q2/GLM-5.3-Flash-Q2.gguf" \
--ssd-streaming \
--ssd-streaming-cache-experts 32GB \
--rocm \
--ctx 500000 \
--tokens 131072 \
--temp 1.0 \
--top-p 0.95 \
--think-max
```

Memory plan:

```text
KV 5.57 GiB
buffers 3.16 GiB
resident model 0.63 GiB
expert cache 28.20 GiB
prefill reserve 3.80 GiB
--------------------------------
total 41.36 GiB
```

Performance:

```text
prefill: 1.24 t/s
generation: 2.48 t/s
```

---

## Test 1B — 500K context / 96 GiB expert cache

DS4's memory guard was disabled:

```bash
DS4_GLM_MEMORY_GUARD=0
```

Command:

```bash
DS4_GLM_MEMORY_GUARD=0 \
./ds4 \
-m "$HOME/models/GLM-5.3-Flash-Q2/GLM-5.3-Flash-Q2.gguf" \
--ssd-streaming \
--ssd-streaming-cache-experts 96GB \
--rocm \
--ctx 500000 \
--tokens 131072 \
--temp 1.0 \
--top-p 0.95 \
--think-max
```

Performance:

```text
prefill: 3.54 t/s
generation: 5.62 t/s
```

Increasing the expert-cache budget from 32 GiB to 96 GiB increased generation throughput from:

```text
2.48 → 5.62 t/s
```

---

## Test 1C — 32K context / maximum practical expert budget

Requested:

```text
--ssd-streaming-cache-experts 110GB
```

DS4 internally capped the budget:

```text
ds4: rocm SSD streaming cache budget 110.00 GiB capped to 101.00 GiB
```

Effective configuration:

```text
total expert budget: 101.00 GiB
prefill headroom: 3.80 GiB
dynamic cache: 81.63 GiB
```

Command:

```bash
DS4_GLM_MEMORY_GUARD=0 \
./ds4 \
-m "$HOME/models/GLM-5.3-Flash-Q2/GLM-5.3-Flash-Q2.gguf" \
--ssd-streaming \
--ssd-streaming-cache-experts 110GB \
--rocm \
--ctx 32000 \
--tokens 2000 \
--temp 1.0 \
--top-p 0.95 \
--think-max
```

Memory plan:

```text
KV 0.36 GiB
buffers 2.98 GiB
resident model 0.63 GiB
expert cache 81.63 GiB
prefill reserve 3.80 GiB
--------------------------------
total 89.39 GiB
```

DS4 reported:

```text
ds4: GLM session ctx=32000 (model max=1048576)
ds4: GLM graph using compact DSA KV only
ds4: GLM graph allocating compact DSA cache:
rows=32000
logical_ctx=32000
kv_layers=45
indexer_layers=11
f16 0.36 GiB
```

Performance:

```text
prefill: 17.96 t/s
generation: 5.96 t/s
```

---

# Test 2 — Unsloth GLM-5.3-Flash UD-Q2_K_XL / llama.cpp / Vulkan

This is a separate model and runtime and is included for comparison.

## Model

Unsloth:

```text
unsloth/GLM-5.3-Flash-GGUF
UD-Q2_K_XL
```

Local model:

```text
$HOME/models/GLM-5.3-Flash-UD-Q2_K_XL-GGUF/
```

The model is approximately **109 GB decimal**, which is about **101.49 GiB**.

## Command

```bash
./llama-server --host 0.0.0.0 --port 11434 \
-m "$HOME/models/GLM-5.3-Flash-UD-Q2_K_XL-GGUF/GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf" \
-dev Vulkan0 -ngl 999 -fa on --jinja --metrics --numa numactl \
--reasoning on --reasoning-format deepseek \
--chat-template-kwargs '{"reasoning_effort":"max"}' \
--timeout 999999 \
-b 512 -ub 256 --cache-reuse 256 \
-n -1 -np 1 -c 500000 \
--temp 1.0 --top-p 0.95
```

Performance observed on the same Strix Halo / Radeon 8060S system:

```text
generation: 9.72 t/s
```

The llama.cpp test used a **500K context**.

---

# Side-by-side results

| Model / Runtime | Backend | Context | Expert cache | Generation |
|---|---|---:|---:|---:|
| Antirez GLM-5.3 Flash Q2 / DS4 | ROCm | 500K | 32 GiB | **2.48 t/s** |
| Antirez GLM-5.3 Flash Q2 / DS4 | ROCm | 500K | 96 GiB | **5.62 t/s** |
| Antirez GLM-5.3 Flash Q2 / DS4 | ROCm | 32K | 81.63 GiB dynamic | **5.96 t/s** |
| **Unsloth GLM-5.3 Flash UD-Q2_K_XL / llama.cpp** | **Vulkan** | **500K** | llama.cpp resident/offload path | **9.72 t/s** |
| **Weschera/glm53-flash-one-spark GLM-5.3 Flash UD-Q2_K_XL / llama.cpp** | **Vulkan** | **500K** | llama.cpp resident/offload path | **11.35 t/s** |

## Key observation

On the same Strix Halo 128 GiB system:

```text
Antirez Q2 + DS4 + ROCm:
~5.6–6.0 t/s

Unsloth UD-Q2_K_XL + llama.cpp + Vulkan:
9.72 t/s
```

The Unsloth/llama.cpp configuration is therefore approximately:

```text
9.72 / 5.96 ≈ 1.63×
```

faster in generation than the best DS4 result obtained so far.

This is notable because DS4 is a specialized GLM runtime with a native ROCm path, while the Unsloth model is running through llama.cpp's Vulkan backend.

## Additional DS4 observations

DS4 enables specialized GLM one-token ROCm kernels:

```text
ds4: ROCm Q8 one-token shared-input kernel enabled through 64 KiB LDS
(in_dim=12288)

ds4: ROCm GLM one-token value projection using wave-parallel Q8 rows
```

However, the GLM model is still configured as:

```text
GLM SSD streaming full resident layers: 0
```

even with a large expert cache.

This means the current DS4 result is **not a fully resident GLM-5.3 Q2 run**.

## Question for DS4 developers

Why is the native DS4 ROCm path on Strix Halo currently limited to approximately **6 t/s generation**, while the same hardware reaches approximately **9.72 t/s** with Unsloth UD-Q2_K_XL through llama.cpp/Vulkan?

Specifically:

1. Is `full resident layers: 0` expected for GLM-5.3 ROCm on Strix Halo?
2. Can GLM-5.3 Flash Q2 run fully resident on a 128 GiB Strix Halo, avoiding SSD streaming?
3. Is distributed layer-slice mode usable on a single Strix Halo to achieve a fully resident execution path?
4. Are there additional `gfx1151`/ROCm kernels or build options that significantly improve one-token decode?
5. Is the current ~6 t/s result expected for Strix Halo, or should substantially higher throughput be possible?