huggingface / huggingface/candle

Problem loading metadata of gguf file

Open
#2,152 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

I encountered an error while executing the example [quantized-phi](https://github.com/huggingface/candle/tree/main/candle-examples/examples/quantized-phi), which I slightly modified. However, I suspect the issue might not be with my modifications.

The problem seems to be related to the function `candle_transformers::models::quantized_llama::ModelWeights::from_gguf`. It appears to be unable to locate the necessary metadata from the model. This is interesting because Hugging Face is able to display the model's metadata correctly.

Here are some screenshots for further reference:

![Error screenshot](https://github.com/huggingface/candle/assets/106385654/28c6947d-fa1d-44ce-b2ee-f7cadd67bc57)

![Hugging Face display](https://github.com/huggingface/candle/assets/106385654/2c1f462e-1dc4-40b1-a465-1e6d99a342d1)

I would appreciate any assistance in resolving this issue. Thank you in advance.

[Full Code](https://gist.github.com/cnlancehu/8efa708f371360f9086a5dd62b01f7cb)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.