glm5_next (GLM-5.3-Flash): prompt cache save always fails — save_safetensors on zero-size array in batched-vision blob store; plus latent hybrid-cache bug in CacheWrapper._num_tokens_in_cache

Nobody has claimed this yet.

### Summary

For `glm5_next` (GLM-5.3-Flash), the batched-vision prompt cache never commits a save: every `commit_pending_save` raises `ValueError: [save_safetensors] Cannot serialize an empty array`, so cross-turn prefix reuse never happens and every conversation turn re-prefills the full prompt. On the same build, a `gemma4` VLM caches normally. There is also a second, latent bug in `CacheWrapper._num_tokens_in_cache` that silently discards snapshots for any hybrid-cache model on that path (details + suggested patch below).

### Environment

- Mac Studio M3 Ultra, 512 GB unified memory, macOS 26.3
- llmster `0.0.12-1`, runtime `mlx-llm-mac-arm64-apple-metal-advsimd@1.11.0`
- **Setup note:** the bundled `mlx_vlm` 0.6.5 was manually upgraded in place to 0.6.17 to get the `glm5_next` architecture (0.6.5 predates it). `mlx` 0.32.0 and `transformers` 5.14.1 satisfy 0.6.17's requirements. The model card's `chat_template.jinja` was also patched (`.0.` numeric-dot subscript → `[0].`) to get past the template parser; that is unrelated to this issue.
- Model: `pipenetwork/GLM-5.3-Flash-MLX-8bit` (334 GB, 8-bit group-64), loaded at `contextLength` 32768–262144 (fails identically at both)

### Error (every turn, from the server log)

```
[cache_io_thread][ERROR]: Failed to commit pending prompt cache save:
Traceback (most recent call last):
File ".../mlx_engine/model_kit/batched_vision/cache_io_thread.py", ...
self._cache_store.commit_pending_save(job.pending_save)
File ".../mlx_engine/model_kit/batched_vision/prompt_cache/cache_store.py", ...
self._blob_store.put(
File ".../mlx_engine/model_kit/batched_vision/prompt_cache/blob_store.py", ...
mx.save_safetensors(buffer, arrays, safetensor_metadata)
ValueError: [save_safetensors] Cannot serialize an empty array ('11.1.1')
```

### Root cause analysis

`glm5_next` is a hybrid: its `make_cache()` (mlx_vlm `glm5_next/language.py`) returns `ArraysCache(size=2)` for the 34 KDA (linear-attention) layers and `CacheList(KVCache(), KVCache())` for the 11 full-attention layers. `cache_store.py` builds the save payload as `dict(tree_flatten([cache.state for cache in record_cache]))`; at least one flattened entry (key `'11.1.1'`) is a zero-size array, which `mx.save_safetensors` refuses. The save raises, nothing is ever stored, every request misses.

Measured impact (same prompt thread, appending 16 tokens per turn at ~20K context): TTFT stays at ~55 s on every turn for glm5_next, vs. dropping from 198 s to 1.1 s for `mlx-community/gemma-4-31b-it-8bit` on the same build (its log shows healthy `Prompt cache: using 52215/52248 tokens from cache`; glm5_next produces no cache lines at all).

Note this is *not* an architectural impossibility: the recurrent KDA state can't be trimmed backward, but it snapshots and resumes forward fine — an alternative MLX server reuses the prefix for this same checkpoint across turns (TTFT ~0 s after the base turn). Only the serialization crashes here. A likely fix shape: skip zero-size arrays at save time, record their key/shape/dtype in the safetensors metadata, and reconstruct them on load — though note `prompt_cache/records.py` also destructures `keys, values = cache.state` in places, which `ArraysCache` state doesn't satisfy.

### Second (latent) bug: `CacheWrapper._num_tokens_in_cache`

```python
for entry in cache:
if hasattr(entry, "offset"):
return entry.offset
return None
```

For a glm5_next-style cache, **no top-level entry has `.offset`** (`ArraysCache` doesn't; `CacheList` holds it on its inner `KVCache`s, reachable via `entry.caches`). The function returns `None`, and `_flush_live_cache()` treats that as unrecoverable and resets the cache without storing a snapshot — silently disabling reuse for any hybrid model served through `CacheWrapper`. Tested one-function fix:

```python
for entry in cache:
if hasattr(entry, "offset"):
return entry.offset
inner = getattr(entry, "caches", None)
if inner:
for c in inner:
if hasattr(c, "offset"):
return c.offset
return None
```

(This alone does not fix the glm5_next case above — that model routes to the batched-vision path — but it removes the same failure mode from the `CacheWrapper` path.)

### Repro

1. Runtime with `mlx_vlm` ≥ 0.6.17, load `pipenetwork/GLM-5.3-Flash-MLX-8bit`
2. Multi-turn chat via `/v1/chat/completions` with a few thousand tokens of context
3. Observe the `cache_io_thread` error on each turn and flat TTFT across turns

Happy to provide fuller logs or run diagnostics on this hardware.

*Diagnosed during appliance work with Claude Code.*

Start with batched_vision/prompt_cache/cache_store.py, blob_store.py, and records.py, then inspect CacheWrapper._num_tokens_in_cache and reproduce the glm5_next multi-turn chat described in the issue. Done means prompt-cache saves no longer fail on the reported hybrid cache, loads resume prefix reuse, and CacheWrapper does not discard hybrid-cache snapshots.

Written by the indexing model from the issue text.