huggingface / huggingface/candle
Cannot load tokenizer from gguf file
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
```
cargo run --bin gguf-tokenizer -- --model /home/wy/.cache/huggingface/hub/models--microsoft--Phi-3-mini-4k-instruct-gguf/snapshots/a64113399c2f6b8ad3e11c394733a2ddadaa7f33/Phi-3-mini-4k-instruct-q4.gguf
```
```
let file = File::open(&gguf_path)
.with_context(|| format!("failed to open GGUF file {}", gguf_path.display()))?;
let mut reader = BufReader::new(file);
let content = gguf_file::Content::read(&mut reader).context("failed to load GGUF metadata")?;
// Build the tokenizer directly from the GGUF metadata (tokens, merges, and post-processing).
let tokenizer =
Tokenizer::from_gguf(&content).context("failed to initialize tokenizer from GGUF")?;
```
```
Error: failed to initialize from GGUF
Caused by:
unsupported model `llama`
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by running the provided gguf-tokenizer command with the Phi-3 GGUF file and inspect the failure from Tokenizer::from_gguf after Content::read loads the metadata. Trace why the GGUF metadata reports an unsupported `llama` model. Done means the tokenizer can be initialized from this file without that error, with coverage for the behavior verified by an appropriate test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 48/100