huggingface / huggingface/candle
Error: cannot find tensor weight
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
- Situation
: I am trying to run the local `mistral-7b-instruct-v0.2.Q4_K_M.gguf` model using the `mistral` example.
- Problem
: I get an error message "`Error: cannot find tensor weight`". The error occurs in the `candle_transformers::models::quantized_mistral::Model::new()` method.
- Guess
: Methods in `candle_transformers::models::quantized_mistral::Model` do not support models in `Mistral-7B-Instruct-v0.2-GGUF` mistral`. Right?
---
FYI, the model runs successfully when I use the `candle_transformers::models::quantized_llama::ModelWeights::from_gguf()` method.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at candle_transformers::models::quantized_mistral::Model::new() and reproduce the error with mistral-7b-instruct-v0.2.Q4_K_M.gguf. Compare its handling with quantized_llama::ModelWeights::from_gguf(), which successfully loads the model, and determine whether Mistral GGUF support is missing. Done means the model loads through the Mistral path or the incompatibility is clearly documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100