pre-tuned kernel not found for model microsoft/BitNet-b1.58-2B-4T
Open
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 40.3k
- Forks
- 3.7k
- PR merge metrics
- No merged PRs in 30d
Description
Using the latest version, I'm trying to setup BitNet following the setup_env.py help instructions:
python .\setup_env.py -help
usage: setup_env.py [-h]
[--hf-repo {1bitLLM/bitnet_b1_58-large,1bitLLM/bitnet_b1_58-3B,HF1BitLLM/Llama3-8B-1.58-100B-tokens,tiiuae/Falcon3-7B-Instruct-1.58bit,tiiuae/Falcon3-7B-1.58bit,tiiuae/Falcon3-10B-Instruct-1.58bit,tiiuae/Falcon3-10B-1.58bit,tiiuae/Falcon3-3B-Instruct-1.58bit,tiiuae/Falcon3-3B-1.58bit,tiiuae/Falcon3-1B-Instruct-1.58bit,microsoft/BitNet-b1.58-2B-4T}]
[--model-dir MODEL_DIR] [--log-dir LOG_DIR] [--quant-type {i2_s,tl2}] [--quant-embd] [--use-pretuned]
Running the following command ran into an error however:
python setup_env.py --hf-repo microsoft/BitNet-b1.58-2B-4T -q i2_s --use-pretuned
ERROR:root:Pretuned kernels not found for model microsoft/BitNet-b1.58-2B-4T
It seems that the microsoft/ was not removed, and so didn't match with "BitNet-b1.58-2B-4T"
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in setup_env.py at the --use-pretuned handling and trace how the --hf-repo value is matched to available pre-tuned kernels. Reproduce the command from the report and verify that microsoft/BitNet-b1.58-2B-4T is recognized instead of producing the missing-kernel error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai-infra-agents
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100