huggingface / huggingface/candle

Cannot load tokenizer from gguf file

Open
#3,527 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

```
cargo run --bin gguf-tokenizer -- --model /home/wy/.cache/huggingface/hub/models--microsoft--Phi-3-mini-4k-instruct-gguf/snapshots/a64113399c2f6b8ad3e11c394733a2ddadaa7f33/Phi-3-mini-4k-instruct-q4.gguf
```
```
let file = File::open(&gguf_path)
.with_context(|| format!("failed to open GGUF file {}", gguf_path.display()))?;
let mut reader = BufReader::new(file);
let content = gguf_file::Content::read(&mut reader).context("failed to load GGUF metadata")?;

// Build the tokenizer directly from the GGUF metadata (tokens, merges, and post-processing).
let tokenizer =
Tokenizer::from_gguf(&content).context("failed to initialize tokenizer from GGUF")?;
```

```
Error: failed to initialize from GGUF

Caused by:
unsupported model `llama`
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by running the provided gguf-tokenizer command with the Phi-3 GGUF file and inspect the failure from Tokenizer::from_gguf after Content::read loads the metadata. Trace why the GGUF metadata reports an unsupported `llama` model. Done means the tokenizer can be initialized from this file without that error, with coverage for the behavior verified by an appropriate test.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.