Slower results on GGUF model

Hi, I have been trying to use MKL with SmolLMv2. I don't get any errors when building, but there are no speedups. The weird thing is that when I run the following code:

```rust
use candle_core::{DType, Device, Tensor};
use candle_nn::VarBuilder;
use candle_transformers::generation::{LogitsProcessor, Sampling};
use hf_hub::{Repo, RepoType, api::sync::Api};
use std::{io::Write, time};

use candle_transformers::models::llama as model;
use model::{Llama, LlamaConfig};

const EOS_TOKEN: &str = "";
const DEFAULT_PROMPT: &str = "My favorite theorem is ";

fn main() -> Result<(), Box> {
let dtype = DType::F16;
let device = Device::Cpu;
let (llama, tokenizer_filename, mut cache, config) = {
let api = Api::new()?;
let model_id = "HuggingFaceTB/SmolLM2-360M".to_string();
println!("loading the model weights from {model_id}");
let api = api.repo(Repo::with_revision(
model_id,
RepoType::Model,
"main".to_string(),
));

let tokenizer_filename = api.get("tokenizer.json")?;
let config_filename = api.get("config.json")?;
let config: LlamaConfig = serde_json::from_slice(&std::fs::read(config_filename)?)?;
let config = config.into_config(false);

let filenames = vec![api.get("model.safetensors")?];
let mut cache = model::Cache::new(true, dtype, &config, &device)?;

let vb = unsafe { VarBuilder::from_mmaped_safetensors(&filenames, dtype, &device)? };
(Llama::load(vb, &config)?, tokenizer_filename, cache, config)
};
let input = Tensor::arange(0u32, 128, &device)?.reshape((1, ()))?;
let start = time::Instant::now();
let logits = llama.forward(&input, 10, &mut cache)?;
dbg!(logits.shape());
dbg!(start.elapsed().as_millis());
Ok(())
}
```

I get the following output:

[src/main.rs:73:5] start.elapsed().as_millis() = 825

But when I run this code:

```rust
use std::fs::File;

use candle_core::quantized::gguf_file;
use candle_transformers::models::quantized_llama::ModelWeights;

fn main() -> Result<(), Box> {
let api = hf_hub::api::sync::Api::new()?;
let repo = api.model("HuggingFaceTB/SmolLM2-360M-Instruct-GGUF".to_string());
let model_path = repo.get("smollm2-360m-instruct-q8_0.gguf")?;

let device = candle_core::Device::Cpu;
let mut file = File::open(model_path)?;
let ct = gguf_file::Content::read(&mut file)?;
let mut model = ModelWeights::from_gguf(ct, &mut file, &device)?;
let start = std::time::Instant::now();
let logits = model.forward(
&candle_core::Tensor::arange(0u32, 128u32, &device)?.reshape((1, ()))?,
0usize,
)?;
dbg!(start.elapsed().as_millis());
Ok(())
}
```

I get:

[src/main.rs:20:5] start.elapsed().as_millis() = 8560

I am not using MKL or any other features in these examples. Why are the results different, and why does the GGUF model take 10 times longer to perform the forward pass? Am I missing something?

Compare the two forward paths in the reported examples, starting with candle_transformers::models::llama::Llama::forward and quantized_llama::ModelWeights::forward. Trace how gguf_file::Content::read and ModelWeights::from_gguf set up the GGUF model, then reproduce both timings on the CPU. Done means identifying and documenting the cause of the roughly tenfold difference, with a focused regression test or benchmark if the repository supports one.

Written by the indexing model from the issue text.