Llava/CLIP Models Not Loading Properly

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [x ] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [ x] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [x ] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [ x] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.

# Expected Behavior

Llava should be loading however following the README example I am getting the following error:
`clip_model_load: total allocated memory: 195.95 MB
Traceback (most recent call last):
File "/Users/furm/projects/watcher/main.py", line 26, in
llm.create_chat_completion(
File "/Users/furm/projects/watcher/venv/lib/python3.9/site-packages/llama_cpp/llama.py", line 2017, in create_chat_completion
return handler(
File "/Users/furm/projects/watcher/venv/lib/python3.9/site-packages/llama_cpp/llama_chat_format.py", line 1049, in __call__
assert (
AssertionError
Exception ignored in:
Traceback (most recent call last):
File "/Users/furm/projects/watcher/venv/lib/python3.9/site-packages/llama_cpp/llama_chat_format.py", line 996, in __del__
File "/Users/furm/projects/watcher/venv/lib/python3.9/site-packages/llama_cpp/_utils.py", line 20, in __enter__
LookupError: unknown encoding: ascii
ggml_metal_free: deallocating`

I'm really not too sure what I'm doing wrong... I'm using the code directly from the README, Both the CLIP model and Llava model are in .gguf format.

# Environment and Context

Using a M2 Macbook Air, Python 3.9.6.