Model only outputs G repeatedly in interactive mode with ggml-model-i2_s.gguf
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 40.3k
- Forks
- 3.7k
- PR merge metrics
- No merged PRs in 30d
Description
When I run run_inference.py in interactive mode using the provided ggml-model-i2_s.gguf from Hugging Face, the model only outputs the character G in a loop, no matter what prompt I use.
Command used
python run_inference.py -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf -p "You are a helpful assistant" -cnv
System info
Ubuntu 22.04
Python 3.9 (Conda env)
CPU only (no AVX support)
Is this expected with the i2_s quantized model? Could this be a tokenizer issue or metadata mismatch?
Thanks for your help!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the provided command with run_inference.py and ggml-model-i2_s.gguf on the stated CPU-only setup. Read how interactive mode handles the model, tokenizer, and metadata; done means identifying the cause of the repeated G output and verifying normal responses with more than one prompt.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100