microsoft / microsoft/BitNet

Error when running python ./utils/convert-helper-bitnet.py models/Llama3-8B-1.58-100B-tokens

Open
#375 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
40.3k
Forks
3.7k
PR merge metrics
No merged PRs in 30d

Description

Unable to quantize Llama3-8B-1.58-100B-tokens, this is the log.

Traceback (most recent call last):

File "D:\Git\BitNet\utils\convert-ms-to-gguf-bitnet.py", line 1855, in
main()

File "D:\Git\BitNet\utils\convert-ms-to-gguf-bitnet.py", line 1841, in main
model = convert_model_names(model, params, args.skip_unknown)

File "D:\Git\BitNet\utils\convert-ms-to-gguf-bitnet.py", line 1591, in convert_model_names
raise ValueError(f"Unexpected tensor name: {name}. Use --skip-unknown to ignore it (e.g. LLaVA)")

ValueError: Unexpected tensor name: model.layers.0.mlp.down_proj.weight_scale. Use --skip-unknown to ignore it (e.g. LLaVA)

Error executing command: D:\Program_Files\MiniConda\envs\bitnet-cpp\python.exe D:\Git\BitNet\utils\convert-ms-to-gguf-bitnet.py D:\Git\BitNet\models\Llama3-8B-1.58-100B-tokens --vocab-type bpe --outtype f32 --concurrency 1 --outfile D:\Git\BitNet\models\Llama3-8B-1.58-100B-tokens\ggml-model-f32-bitnet.gguf

Return code: 1

An error occurred: Command '['D:\Program_Files\MiniConda\envs\bitnet-cpp\python.exe', 'D:\Git\BitNet\utils\convert-ms-to-gguf-bitnet.py', 'D:\Git\BitNet\models\Llama3-8B-1.58-100B-tokens', '--vocab-type', 'bpe', '--outtype', 'f32', '--concurrency', '1', '--outfile', 'D:\Git\BitNet\models\Llama3-8B-1.58-100B-tokens\ggml-model-f32-bitnet.gguf']' returned non-zero exit status 1.

Cleaning up intermediate files...

Removing preprocessed file: D:\Git\BitNet\models\Llama3-8B-1.58-100B-tokens\model.safetensors

Restoring original 'D:\Git\BitNet\models\Llama3-8B-1.58-100B-tokens\model.safetensors' from 'D:\Git\BitNet\models\Llama3-8B-1.58-100B-tokens\model.safetensors.backup'

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with utils/convert-ms-to-gguf-bitnet.py, especially convert_model_names around the reported failure at line 1591, and reproduce the command from the issue. Trace how model.layers.0.mlp.down_proj.weight_scale is handled; done means the Llama3-8B-1.58-100B-tokens conversion completes without the Unexpected tensor name error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.