microsoft / microsoft/BitNet

Converting HF model to GGUF doesn't seem to work?

Open
#231 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
40.3k
Forks
3.7k
PR merge metrics
No merged PRs in 30d

Description

I tried to follow the help instructions given by the setup_env.py script and ran the following command:

python setup_env.py --hf-repo microsoft/BitNet-b1.58-2B-4T -q i2_s
INFO:root:Compiling the code using CMake.
INFO:root:Downloading model microsoft/BitNet-b1.58-2B-4T from HuggingFace to models\BitNet-b1.58-2B-4T...
INFO:root:Converting HF model to GGUF format...
ERROR:root:Error occurred while running command: Command '['C:\\Users\\usr\\anaconda3\\envs\\bitnet-cpp\\python.exe', 'utils/convert-hf-to-gguf-bitnet.py', 'models\\BitNet-b1.58-2B-4T', '--outtype', 'f32']' returned non-zero exit status 1., check details in logs\convert_to_f32_gguf.log

error log:

INFO:hf-to-gguf:Loading model: BitNet-b1.58-2B-4T
Traceback (most recent call last):
  File "C:\Users\usr\Desktop\git\BitNet2\utils\convert-hf-to-gguf-bitnet.py", line 1165, in <module>
    main()
  File "C:\Users\usr\Desktop\git\BitNet2\utils\convert-hf-to-gguf-bitnet.py", line 1143, in main
    model_class = Model.from_model_architecture(hparams["architectures"][0])
  File "C:\Users\usr\Desktop\git\BitNet2\utils\convert-hf-to-gguf-bitnet.py", line 240, in from_model_architecture
    raise NotImplementedError(f'Architecture {arch!r} not supported!') from None
NotImplementedError: Architecture 'BitNetForCausalLM' not supported!

I'm using the latest version of the repository code as of today.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with setup_env.py and the failing invocation of utils/convert-hf-to-gguf-bitnet.py, then inspect convert_to_f32_gguf.log and the Model.from_model_architecture call. Reproduce the command with microsoft/BitNet-b1.58-2B-4T and confirm the conversion completes and produces the expected GGUF output.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.