abetlen / abetlen/llama-cpp-python

ValueError: Failed to create llama_context

Open
#1,304 10 comments 5 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10.6k
Forks
1.4k
PR merge metrics
PR metrics pending

Description

(yuna2) (base) adm@Adms-MacBook-Pro yuna-ai % python index.py
ggml_metal_init: error: Error Domain=MTLLibraryErrorDomain Code=3 "program_source:3:10: fatal error: 'ggml-common.h' file not found
#include "ggml-common.h"
^~~~~~~~~~~~~~~
" UserInfo={NSLocalizedDescription=program_source:3:10: fatal error: 'ggml-common.h' file not found
#include "ggml-common.h"
^~~~~~~~~~~~~~~
}
llama_new_context_with_model: failed to initialize Metal backend
Traceback (most recent call last):
File "/Users/adm/Desktop/yuna-ai/index.py", line 171, in
yuna_server = YunaServer()
^^^^^^^^^^^^
File "/Users/adm/Desktop/yuna-ai/index.py", line 33, in __init__
self.chat_generator = ChatGenerator(self.config)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/adm/Desktop/yuna-ai/lib/generate.py", line 11, in __init__
self.model = Llama(
^^^^^^
File "/usr/local/lib/python3.12/site-packages/llama_cpp/llama.py", line 328, in __init__
self._ctx = _LlamaContext(
^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/llama_cpp/_internals.py", line 265, in __init__
raise ValueError("Failed to create llama_context")
ValueError: Failed to create llama_context
Screen Shot 2024-03-25 at 23 43 18

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.