'list' object has no attribute 'swapaxes' — deterministic cache-rebuild crash on GLM-4.7-Flash (glm4_moe_lite MLA) under Parallel>1

## Summary

LM Studio's MLX engine enters a deterministic corrupted state while serving `zai-org/glm-4.7-flash` (`glm4_moe_lite`, 6-bit MLX, MLA attention) when the model is loaded with `Parallel: 4` and a client reuses prompt caching across requests. Once corrupted, every subsequent `/v1/chat/completions` request returns:

```json
{"error":"Error in iterating prediction stream: AttributeError: 'list' object has no attribute 'swapaxes'"}
```

The only recovery is `lms unload` + `lms load`. We reproduced this across two independent BigCodeBench (N=200) runs against the same loaded instance. Both runs served tasks successfully up to `BigCodeBench/35` and then cascaded on `BigCodeBench/36` with identical error strings and no recovery. A trivial client-side mitigation (`"cache_prompt": false`) makes the crash go away completely, which strongly points at the cross-request MLA cache reuse path as the root cause.

## Error signature

```
Error in iterating prediction stream: AttributeError: 'list' object has no attribute 'swapaxes'
```

HTTP response: `400 Bad Request` on `/v1/chat/completions`. Once the state is corrupted, every subsequent request fails with this string (hundreds of rows in our telemetry), and the only recovery is unload/reload of the model.

## Environment

| Field | Value |
|---|---|
| LM Studio | `0.4.9+1` (CLI commit `8d3f370`) |
| MLX runtime | `mlx-llm-mac-arm64-apple-metal-advsimd@1.5.0` |
| `mlx-engine` commit | `125c501` |
| `mlx` | `0.31.1` |
| `mlx-lm` | `0.31.2` |
| Hardware | Apple M4 Max, 128 GB unified memory |
| OS | macOS 26.4 (Darwin 25.4.0, arm64) |
| Model | `zai-org/glm-4.7-flash` |
| Architecture | `glm4_moe_lite` (MoE reasoning, MLA attention) |
| Quantization | 6-bit MLX |
| `loaded_context_length` | 66000 |
| `max_context_length` | 202752 |
| `lms ps` output | `Parallel: 4`, 24.36 GB, TTL 20 min |
| Endpoint | `http://:8200/v1/chat/completions` (REST API) |
| Client concurrency | `1` (single-threaded HTTP, one request in flight) |
| Benchmark workload | BigCodeBench direct, `enable_thinking=true` |

## Reproduction

We do not have a minimal standalone repro script. What we observed is reproducible from the following setup, and we believe any reasoning-mode long-run workload against this model will trigger it.

1. On a LAN host, load the model with default `Parallel: 4`:

```bash
lms load zai-org/glm-4.7-flash --context-length 66000
# defaults to Parallel: 4 in our install
```

2. Leave prompt caching at default (cache enabled) — do NOT pass `"cache_prompt": false`.

3. From a single-threaded client (concurrency=1), hit `/v1/chat/completions` with reasoning-mode ("thinking") enabled and moderately long completions (1,000–3,400 tokens per request) against a stream of distinct prompts that share some common system-prefix.

4. Serve ~35–40 such requests sequentially. Accumulated completion tokens across all successful requests land somewhere in the `~43K–67K` range.

5. The next request that requires a full MLA cache rebuild (because the common-prefix trim across successive requests produces a large delta for the first time since model load) fails with:

```
'list' object has no attribute 'swapaxes'
```

6. Every subsequent request then fails with the same error until `lms unload` + `lms load`.

Our exact workload: BigCodeBench direct, 200 tasks, `enable_thinking=true`, single-threaded Python HTTP client, no prompt caching opt-out.

## Evidence — deterministic break point

Both N=200 BigCodeBench runs, on the same loaded instance of `zai-org/glm-4.7-flash`, broke at identical task indices. Data from our benchmark warehouse (`performance_records` table):

| Run (prefix) | OK tasks before cascade | Last OK task | First permanent-cascade task | Cumulative completion tokens pre-break | Cumulative prompt tokens pre-break | Wall clock at break |
|---|---|---|---|---|---|---|
| `1b6be49f…` | 16 (of 36) | `BigCodeBench/35` | `BigCodeBench/36` | 43,614 | 7,092 | T+16 min |
| `b4ee0124…` | 21 (of 36) | `BigCodeBench/35` | `BigCodeBench/36` | 67,346 | 10,927 | T+23 min |

Key observations:

- **Both runs last-OK = `BigCodeBench/35`, both runs first-permanent-cascade = `BigCodeBench/36`.** This is deterministic across two independent runs against the same loaded instance.
- **No per-call token ceiling.** Run 2 served a 3,367-completion-token task successfully at `BigCodeBench/10`, well after the point where run 1's `/36` would have fired. Larger individual completions ran fine both before and after the break point. It is not a "request too big" issue.
- **No latency ceiling.** All successful calls completed well under 70 seconds; our client-side `task_timeout` is 300 seconds. The 400s come back in ~20 ms after the corruption, not from a timeout.
- **Cumulative completion tokens at break ≈ 43 K (run 1) / 67 K (run 2).** The loaded context length is 66,000. The break fires right around the first moment that the cross-request cache delta (from `cache_wrapper.py` common-prefix computation) exceeds the context-length boundary and triggers a full MLA cache rebuild for the first time.
- Run 1 also had an earlier transient cascade at tasks `/8`–`/18` (self-recovered at `/19`). Run 2 had a brief blip at `/21`–`/23` (also self-recovered). Both runs then hit the permanent cascade at `/36`. The transient blips are consistent with narrower common-prefix deltas that self-healed on the next request; the permanent cascade at `/36` is when the MLA cache state passes the point of no return.

## Root cause hypothesis

We traced the crash to `mlx_lm/models/glm4_moe_lite.py:151-153` on the installed runtime:

```python
if cache is not None:
kv_latent, k_pe = cache.update_and_fetch(kv_latent, k_pe)
pe_scores = (q_pe * self.scale) @ k_pe.swapaxes(-1, -2)
```

`k_pe` is expected to be an `mx.array`. The only call site that can substitute a list is `cache.update_and_fetch(kv_latent, k_pe)`. `glm4_moe_lite.py` does not define a custom `make_cache`, so `make_prompt_cache` (`mlx_lm/models/cache.py`) returns the default `KVCache` / `RotatingKVCache`. Neither of those returns Python lists under steady-state generation.

The substitution appears to happen inside LM Studio's cross-request prompt-cache reuse layer, `mlx_engine/cache_wrapper.py` (roughly lines 122–168 in the version we have installed), which:

1. Computes common-prefix length across successive requests.
2. Calls `trim_prompt_cache(cache, num_tokens_to_trim)` — defined in `mlx_lm/models/cache.py` as `[c.trim(n) for c in cache][0]`, which returns an int, not a cache.
3. Falls through to `make_prompt_cache()` on partial trim.

For an MLA cache (where `keys` / `values` are overloaded as `kv_latent` / `k_pe`), a mid-rebuild exception in step 2 appears to leave `cache[layer].state` holding raw Python list data — which matches the `'list' object has no attribute 'swapaxes'` traceback from `glm4_moe_lite.py:153` exactly. Once corrupted, the cache object is reused on every subsequent request and every request fails the same way.

The fact that `Parallel: 4` is set on the instance activates the MLX runtime's `BatchKVCache` continuous-batching path (shipped with MLX runtime 1.0.0). The closed-but-related PR [ml-explore/mlx-lm#798](https://github.com/ml-explore/mlx-lm/pull/798) ("BatchKVCache filter crash on reuse") fixed a similar cross-request reuse crash for other model architectures; we suspect this `glm4_moe_lite` MLA path is a structurally analogous bug that was not covered by that fix.

Empirical validation of the hypothesis: setting `"cache_prompt": false` on every HTTP request — which forces `make_prompt_cache(...)` to run fresh per request and disables cross-request cache reuse entirely — eliminates the crash completely across multi-hour load in our environment. This is consistent with the root cause being in the cross-request MLA cache reuse / trim / rebuild transition, not in the per-request forward pass.

### Related upstream references (for context — none match this exact string)

| Ref | Relevance |
|---|---|
| [ml-explore/mlx-lm#777 — GLM 4.7 Flash bug](https://github.com/ml-explore/mlx-lm/issues/777) | Same model, coherence degradation on MLX q4. Closed. |
| [ml-explore/mlx-lm#780 — Store KV latent in cache](https://github.com/ml-explore/mlx-lm/pull/780) | Introduced the exact `k_pe.swapaxes(-1,-2)` line that crashes for us. Merged. |
| [ml-explore/mlx-lm#790 — GLM 4.7 Flash KV Cache size](https://github.com/ml-explore/mlx-lm/issues/790) | Same model, oversized KV cache. Closed. |
| [ml-explore/mlx-lm#798 — BatchKVCache filter crash on reuse](https://github.com/ml-explore/mlx-lm/pull/798) | Structural analog — "reuse prompt cache across runs crashes" under Parallel>1. Merged. Our bug looks like the same class, unfixed for MLA. |
| [ml-explore/mlx-lm#877 — Missing `<|think|>` tag for GLM-4.7-Flash](https://github.com/ml-explore/mlx-lm/issues/877) | Open, different symptom. |
| [lmstudio-ai/mlx-engine#106](https://github.com/lmstudio-ai/mlx-engine/issues/106), [#136](https://github.com/lmstudio-ai/mlx-engine/issues/136) | Same error *prefix* ("Error in iterating prediction stream") but different root causes. |
| [lmstudio-ai/lmstudio-bug-tracker#1390](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1390), [#1504](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1504) | Same model, general instability, different symptom. |
| [ggml-org/llama.cpp#19068](https://github.com/ggml-org/llama.cpp/issues/19068) | Identical semantic pattern ("GLM-4.7-Flash enters corrupted state") on a different backend. Suggests this class of cache-state bug exists across inference engines for this model. |

**We could not find any existing issue on `lmstudio-ai/mlx-engine` or `ml-explore/mlx-lm` that references the exact string `'list' object has no attribute 'swapaxes'`.**

## Mitigations we're using

We've deployed all four of these in our benchmarking harness. Layers 1 and 4 are the two that actually prevent or contain the bug; 2 and 3 are defense-in-depth for residual failure modes.

1. **Proactive — `"cache_prompt": false` on every LM Studio HTTP request.** This forces a fresh prompt cache per request and prevents the cross-request MLA cache state from ever accumulating into the corrupted trim/rebuild transition. Since deploying this, we've held zero errors across a multi-hour GLM-4.7-Flash load test. This is the single most effective mitigation.
2. **Reactive retry — on `400` with `swapaxes` in the body, retry once with `enable_thinking=false`.** Catches any edge case the proactive layer misses. Logs provenance so we know when it fired.
3. **Circuit breaker — benchmark harness stops after 5 consecutive identical API errors.** Prevents a single cache corruption from contaminating an entire run.
4. **Operational — `lms load ... --parallel 1` on the affected host.** Orthogonal mitigation. Disables the `BatchKVCache` continuous-batching path entirely. Recommended for long stability-sensitive runs regardless of benchmark.

Of these, **(1) `cache_prompt: false`** is the one that actually keeps the bug from ever firing. It's also the one that is cheapest for the user to apply while waiting for an upstream fix, and we'd suggest this as an interim workaround for anyone else who hits this.

## Suggested upstream direction

We don't want to overstep, but a few directions that might help:

- In `mlx_engine/cache_wrapper.py`, guard the cross-request cache reuse path so that any exception during `trim_prompt_cache` on an MLA-latent cache falls back to a fresh `make_prompt_cache(...)` and discards the partially-rebuilt state, rather than leaving `cache[layer].state` holding a Python list.
- Add a type assertion / runtime check on the `keys` / `values` fields of MLA-style caches before they hit the attention kernel, so the failure surfaces at the cache boundary rather than deep inside `glm4_moe_lite.py:153`.
- Consider making `cache_prompt: false` the default for `glm4_moe_lite` models until the underlying MLA cache reuse path is verified against the `BatchKVCache` continuous-batching code path (analogous to the fix in [ml-explore/mlx-lm#798](https://github.com/ml-explore/mlx-lm/pull/798) for non-MLA architectures).

## Additional context

This was discovered during a large benchmarking campaign (BigCodeBench, N=200, reasoning mode) against `zai-org/glm-4.7-flash` on LM Studio. We have a full forensic investigation report including DuckDB queries, token histograms, cache-wrapper code traces, and the complete list of upstream issues we checked. Happy to share the full internal investigation privately if that would be useful to a maintainer — just ask.

If it helps, we can also:

- Run a bisection against different `lms load --parallel` values (1 / 2 / 4) on the same workload to confirm the `BatchKVCache` hypothesis.
- Re-run with `enable_thinking=false` on the same cache-enabled path to test whether reasoning mode is strictly necessary to trigger the crash.
- Share our anonymized benchmark task IDs and observed completion-token distributions so a maintainer could build a minimal repro harness.

Start with mlx_engine/cache_wrapper.py, especially the cross-request trim and rebuild path, then compare it with mlx_lm/models/cache.py and glm4_moe_lite.py:151-153. Reproduce with Parallel: 4, prompt caching enabled, and the sequential reasoning workload described in the issue; verify that repeated requests no longer enter the permanent 'list' object has no attribute 'swapaxes' state and that cache_prompt=false remains unnecessary.

Written by the indexing model from the issue text.