Unable to disable "clip_model_load" log messages

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [X] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [X] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [X] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [X] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.

# Expected Behavior

verbose=false passed to Llama should disable log messages for llama_cpp

# Current Behavior

Log messages are leaking through from an underlying llama_chat_format.

```
clip_model_load: loaded meta data with 18 key-value pairs and 377 tensors from models/llava/mmproj-model-f16.gguf
clip_model_load: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
clip_model_load: - kv 0: general.architecture str = clip
...
```

# Environment and Context

M1 Pro - MacBook Pro

```
$ python3 --version
Python 3.10.13
$ make --version
GNU Make 3.81
$ g++ --version
Apple clang version 15.0.0 (clang-1500.1.0.2.5)
```

# Failure Information (for bugs)

I believe verbose should suppress these log messages.

# Steps to Reproduce

Use the following code:

```
from llama_cpp import Llama
from llama_cpp.llama_chat_format import Llava15ChatHandler

import logging
logger = logging.getLogger('llama_cpp.llama_chat_format')
logger.disabled = True

def load_llm():
chat_handler = Llava15ChatHandler(clip_model_path="./models/llava/mmproj-model-f16.gguf")

llm = Llama(
model_path="./models/llava/ggml-model-q5_k.gguf",
chat_handler=chat_handler,
verbose=False,
n_ctx=1024, # n_ctx should be increased to accommodate the image embedding
logits_all=True, # needed to make llava work
)
return llm

def run(input, llm):
response = llm.create_chat_completion(
messages=[
{"role": "system", "content": "You are an assistant who perfectly describes images."},
{
"role": "user",
"content": [
input,
{
"type": "text",
"text": "Describe this image in detail please."
}
]
}
]
)

return response['choices'][0]['message']['content']

if __name__=='__main__':
input = {
"type": "image_url",
"image_url": {"url": "https://thumbor.forbes.com/thumbor/fit-in/900x510/https://www.forbes.com/advisor/wp-content/uploads/2023/07/top-20-small-dog-breeds.jpeg.jpg"}
}

llm = load_llm()
response = run(input, llm).strip()

print(response)
```