i2_s SIGSEGVs at n_ubatch >= 32: BLAS backend dequantises by a row stride 4x the real packed row

Summary

Any i2_s model SIGSEGVs during prompt processing as soon as n_ubatch >= 32, on any build that links a BLAS backend — which is the default on macOS via Accelerate. Single-token generation is unaffected, which is why this doesn't show up in tg benchmarks.

Repro

BitNet-b1.58-2B, i2_s:

llama-bench -m ggml-model-i2_s.gguf -ngl 0 -p 64 -ub 32   # SIGSEGV
llama-bench -m ggml-model-i2_s.gguf -ngl 0 -p 64 -ub 16   # fine

The threshold is exactly min_batch = 32 in ggml_backend_blas_device_supports_op — a backend-selection boundary, not a stack limit.

Cause

i2_s packs four 2-bit weights per byte but declares blck_size = 1, type_size = 1 (ggml.c, [GGML_TYPE_I2_S]). So

result->nb[1] = result->nb[0] * (result->ne[0] / ggml_blck_size(type));   // = ne0 bytes

while a real packed row is ne0/4. The row stride is intentionally 4x too large, and every BitNet-aware consumer compensates at the use site — src0_row + ir0 * nb01 / 4 appears verbatim three times in ggml_compute_forward_mul_mat. ggml_nbytes carries the matching correction (nbytes / 4 + 32).

That convention is self-consistent until a generic path touches i2_s. ggml_backend_blas_device_supports_op accepts any src0 whose type has a non-NULL to_float:

(src0->type == GGML_TYPE_F32 || ggml_get_type_traits(src0->type)->to_float != NULL);

i2_s qualifies, and the backend then dequantises row-wise by the raw nb01, walking four times too far and off the end of the tensor.

There is also no correct i2_s BLAS path even in principle: dequantize_row_i2_s takes a fourth argument (the scale) and is cast to the three-argument ggml_to_float_t, so the scale is never passed. Even if the stride fit, the dequantised weights would be unscaled — and BLAS has no way to apply the act_sums / +1-offset correction that the generic mul_mat wrapper applies for these types.

Fix

Exclude I2_S/TL1 from the BLAS backend and let the generic per-row path handle them, which it already does correctly. Two-line guard in the GGML_OP_MUL_MAT case. I'll open a PR.

Measured after the fix (M5 Max, -ngl 0), comparing against -ub 16, previously the only working setting:

Correctness, not just non-crash: greedy (--temp 0 --seed 1), the same prompt at -ub 16 / -ub 32 / -ub 512 produces byte-identical output, and the model correctly answers a fact-recall question that requires a 46-token prompt to have been prefilled correctly. -ub 16 is the pre-existing known-good path, so identity against it is the meaningful check. Verified with GGML_BLAS both ON and OFF.

Verification note

The bug is confirmed present at the currently pinned submodule commit 390c3077 by source inspection — min_batch = 32 and the to_float != NULL qualification are intact at ggml-blas.cpp:416-422, and blck_size = 1 plus the 4-argument to_float cast are intact at ggml.c:931-936.

The fix was built and measured against the earlier pinned tree 1f86f058 (build 3962), because current main does not build on Apple silicon for unrelated reasons. I have not been able to run current main end-to-end, so I'd appreciate a sanity check from someone who can.

Start in ggml-blas.cpp at ggml_backend_blas_device_supports_op and the GGML_OP_MUL_MAT case, then inspect the I2_S traits and dequantization details in ggml.c. Reproduce with llama-bench at -ub 32 and -ub 16; done means the generic path handles I2_S/TL1 without a crash and output remains byte-identical across batch sizes.