ARM TL1 memory error: "double free or corruption (!prev)" and "free(): invalid next size (normal)"

I encountered this on Orange Pi 5 Plus (RK3588) with Ubuntu 22.04 LTS. The compilation environments (in a Conda env) are:

• Python 3.9.21
• CMake 3.31.2
• Clang 18.1.8

The compilation went successfully, but both run_inference.py (llama-cli) and e2e_benchmark.py (llama-bench) exited with some memory errors. Specifically in the benchmark case, there are still benchmark outputs, but I'm not sure will this affect the measured performance.

Note that there are different errors when benchmarking prefilling (p>0, n=0) and decoding (p=0, n>0).

• "double free or corruption (!prev)" in prefilling
• "free(): invalid next size (normal)" in decoding

Here are the corresponding commands and outputs:

(bitnet-cpp) orangepi@orangepi5plus:~/repos/BitNet$ python utils/e2e_benchmark.py -m ~/models/bitnet_b1_58-3B/ggml-model-tl1.gguf -n 0 -p 128 -t 4
| model                          |       size |     params | backend    | threads | n_batch |          test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | ------------: | -------------------: |
| bitnet 3B TL1                  | 965.22 MiB |     3.32 B | CPU        |       4 |       1 |         pp128 |         13.88 ± 0.07 |
double free or corruption (!prev)
ERROR:root:Error occurred while running command: Command '['/home/orangepi/repos/BitNet/build/bin/llama-bench', '-m', '/home/orangepi/models/bitnet_b1_58-3B/ggml-model-tl1.gguf', '-n', '0', '-ngl', '0', '-b', '1', '-t', '4', '-p', '128', '-r', '5']' died with <Signals.SIGABRT: 6>.

(bitnet-cpp) orangepi@orangepi5plus:~/repos/BitNet$ python utils/e2e_benchmark.py -m ~/models/bitnet_b1_58-3B/ggml-model-tl1.gguf -n 32 -p 0 -t 4
| model                          |       size |     params | backend    | threads | n_batch |          test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------: | ------------: | -------------------: |
| bitnet 3B TL1                  | 965.22 MiB |     3.32 B | CPU        |       4 |       1 |          tg32 |         14.43 ± 0.19 |
free(): invalid next size (normal)
ERROR:root:Error occurred while running command: Command '['/home/orangepi/repos/BitNet/build/bin/llama-bench', '-m', '/home/orangepi/models/bitnet_b1_58-3B/ggml-model-tl1.gguf', '-n', '32', '-ngl', '0', '-b', '1', '-t', '4', '-p', '0', '-r', '5']' died with <Signals.SIGABRT: 6>.

This should be a similar issue to https://github.com/microsoft/BitNet/issues/143, but I'm not quite sure.

Reproduce the failure with utils/e2e_benchmark.py using the two llama-bench commands shown, and compare the symptoms with issue 143. Inspect the llama-bench and inference entry points involved by those commands, then use the prefilling and decoding cases to narrow the failure. Done means both benchmarks complete without allocator errors or SIGABRT and their outputs remain valid.