llama2 benchmark copies carry the four numerical defects fixed in 997d3d87

benchmarks/llama2_100.vx and benchmarks/llama2_scaling.vx carry all four of the numerical defects fixed in tests/backend/pass/llama2.vx by 997d3d87. Any timing these files have produced describes a model that is not Llama.

The defects, as they appear in the benchmark copies

The fixes are mechanical and are exactly those in 997d3d87: derive dim_f, hidden_dim_f and head_size_f from the config with as f32 (available since hiraditya/Vx.1#240), compute head_dim = i % head_size before calling vx_get_rope_freq, and split the softmax into let shifted = val - max_val; shifted.exp().

Why this matters beyond tidiness

These are the files behind cargo vx-bench and the llama2 scaling numbers. The arithmetic cost per token is unchanged by the fixes — same matmul shapes, same loop trip counts — so previously published timings are not invalidated in magnitude. What is invalidated is any claim that the benchmark runs Llama inference: it runs a transformer with a broken softmax and a scrambled positional encoding, and produces degenerate text.

Two consequences to decide on:

• If any benchmark result has been quoted anywhere as "Llama2 inference in Vx", it should be re-run after the fix so the claim and the computation agree.
• benchmarks/ane_decode_bench.vx:199 passes a variable named head_dim to vx_get_rope_freq; worth confirming it is the within-head index and not the head size.

Suggested approach

Rather than patching three divergent copies of the same transformer, consider whether the benchmark files can import the kernels from one place. Four copies of this forward pass now exist (tests/backend/pass/llama2.vx, llama2_v2.vx, examples/llama.vx, and the two benchmarks), and they disagree with each other: llama2_v2.vx and examples/llama.vx compute the RoPE modulo correctly while the others do not. That divergence is the actual defect generator.

Verification bar for the fix: token-level parity against llama2.c on stories15M with greedy sampling from BOS, the same check 997d3d87 used (64/64 token ids identical).

Start by comparing benchmarks/llama2_100.vx and benchmarks/llama2_scaling.vx with the corrected implementation in tests/backend/pass/llama2.vx and commit 997d3d87. Apply the corresponding numerical fixes, inspect benchmarks/ane_decode_bench.vx:199, and verify 64/64 token-id parity against llama2.c on stories15M with greedy sampling from BOS.