Web Server GPU Usage Anomaly

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [Y] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [Y] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [Y] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [Y] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.

# Expected Behavior

That the webserver would utilise 100% of the GPU (or close to it).

# Current Behavior

Currently it is only using 25% of each GPU it never really goes beyond this:


# Environment and Context

This is Linux Ubuntu running latest version of Llama-cpp-python with 2x A100 GPU (all the cuda is up to date, cblas is on etc. This works at 100% using vLLM - so I don't believe there is any issue with the hardware.

# Steps to Reproduce

Please provide detailed steps for reproducing the issue. We are not sitting in front of your screen, so the more detail the better.

I am running the web server using the command: `python -m llama_cpp.server --config_file serverconfig.json`

My config file is below, I show this because I know I had a problem with GPU usage when not using `offload_kqv = true` previously and wanted to demonstrate that I was aware of this and my config has this set as true.

If any paths, or anything looks odd in the syntax, that is because I have anonymised it - the server runs, it accepts requests, it responds all of that is fine, it is just the utilisation that is a problem.

```
{
"host": "0.0.0.0",
"port": 8000,
"models": [
{
"model": "/models/mistral-7b-instruct-v0.2.Q5_K_M.gguf",
"n_gpu_layers": -1,
"offload_kqv": true,
"n_threads": 12,
"n_batch": 512,
"n_ctx": 8192
}
]
}
```

This loads up the two A100 GPU, it recognises them, offloads 33 layers to them and provides this response:


I have also run this same machine using vLLM's server. It utilises 100% of the GPUs both of them at times. So I know it is possible to get them both to 100% utilisation, but I can't seem to get there with the llama-cpp-python web server.

I was hoping that this is a me problem, that I have not set a flag I need to set or that the 25% being recorded is actually the full utilisation but this is an Nvidia thing that interacts with Llama-cpp this way, but doesn't using whatever vLLM is using...

Just thought I would add, this is the GPU usage when running vLLM: