huggingface / huggingface/candle

Error: cannot find tensor weight

Open
#1,630 4 comments 5 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

- Situation
: I am trying to run the local `mistral-7b-instruct-v0.2.Q4_K_M.gguf` model using the `mistral` example.

- Problem
: I get an error message "`Error: cannot find tensor weight`". The error occurs in the `candle_transformers::models::quantized_mistral::Model::new()` method.

- Guess
: Methods in `candle_transformers::models::quantized_mistral::Model` do not support models in `Mistral-7B-Instruct-v0.2-GGUF` mistral`. Right?

---

FYI, the model runs successfully when I use the `candle_transformers::models::quantized_llama::ModelWeights::from_gguf()` method.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at candle_transformers::models::quantized_mistral::Model::new() and reproduce the error with mistral-7b-instruct-v0.2.Q4_K_M.gguf. Compare its handling with quantized_llama::ModelWeights::from_gguf(), which successfully loads the model, and determine whether Mistral GGUF support is missing. Done means the model loads through the Mistral path or the incompatibility is clearly documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.