llama-server ignores bitnet-b1.58.rope.freq_base metadata, causing NaN in softmax. llama-cli works correctly.
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 40.3k
- Forks
- 3.7k
- PR merge metrics
- No merged PRs in 30d
Description
Evidence:
- GGUF metadata: bitnet-b1.58.rope.freq_base = 500000
- Server crashes at ggml.c:14652 (assert !isnan) regardless of --rope-freq-base flag
- Same model/prompt works fine with llama-cli
- Crash appears twice simultaneously (multi-thread confirm)
- GDB backtrace: server_context::update_slots() → llama_decode()
→ ggml_compute_forward_soft_max_f32() → __assert_fail - llama-cli works because it doesn't use server_context slot management
Command used:
build/bin/llama-server -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf -c 4096 -t 6 -n 1024 -ngl 0 -b 1 --temp 0.8 --rope-freq-base 500000 --host 127.0.0.1 --port 8080
Error:
Segmentation Fault
llama-server: /home/user/Documentos/Codigos/bitnet/bitnet_code/3rdparty/llama.cpp/ggml/src/ggml.c:14652: void ggml_compute_forward_soft_max_f32(const struct ggml_compute_params *, struct ggml_tensor *): Assertion `!isnan(wp[i])' failed.
llama-server: /home/user/Documentos/Codigos/bitnet/bitnet_code/3rdparty/llama.cpp/ggml/src/ggml.c:14652: void ggml_compute_forward_soft_max_f32(const struct ggml_compute_params *, struct ggml_tensor *): Assertion `!isnan(wp[i])' failed.
Prompt used:
---------------------
Context information is below.
[1] the positions of Russian troops and military equipment. Information from partisans helped the Ukrainian army destroy a Russian warehouse in Nova Kakhovka on July 11. In September -October 2002, Ukrainian forces intensively destroyed Russian facilities and personnel in the Kherson region, precipita ting the rapid liberation of Kherson city and the entire Mykola yiv region in November 2022. Conclusion Russian authorities have not been able to suppress Ukraine’s partisan movements or
---------------------
Query: Tell me everything you know about partisan warfare in occupied ukraine
Machine:
OS: Debian GNU/Linux 12 (bookworm) x86_64
Kernel: 6.1.0-44-amd64
CPU: Intel i7-8665U (8) @ 4.800GHz
GPU: NVIDIA GeForce MX250
GPU: Intel WhiskeyLake-U GT2 [UHD Graphics 620]
Memory: 4125MiB / 7564MiB
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the supplied llama-server command and compare it with llama-cli using the same model and rope frequency setting. Trace server_context::update_slots() into llama_decode() and ggml_compute_forward_soft_max_f32(), using the stacktrace and the reported bitnet-b1.58.rope.freq_base metadata as context. Done means llama-server handles the model without the NaN assertion and preserves the working llama-cli behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100