abetlen / abetlen/llama-cpp-python
I'm loading a gguf but llama-python-cpp says "is it really a GGML file" ???
- 主要語言
- Python
- 星號
- 10.6k
- 分支
- 1.4k
- PR 合併指標
- PR 指標待擷取
描述
Installed llama_cpp_python-0.2.43.tar.gz via
CMAKE_ARGS="-DLLAMA_CUBLAS=ON -DCMAKE_CUDA_COMPILER=/opt/cuda/bin/nvcc -DTCNN_CUDA_ARCHITECTURES=61" pip install llama-cpp-python
llm = Llama(model_path="/home/takosalad/llama/models/nous-capybara-3b-v1.9.q5_k_m.gguf",
n_gpu_layers=63, n_threads=6, n_ctx=3584, n_batch=512, verbose=True),
ggml_init_cublas: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9
llama.cpp: loading model from /home/takosalad/llama/models/nous-capybara-3b-v1.9.q5_k_m.gguf
error loading model: unknown (magic, version) combination: 46554747, 00000003; is this really a GGML file?
llama_load_model_from_file: failed to load model
Traceback (most recent call last):
File "/home/takosalad/llama/./test.py", line 35, in
llm = Llama(model_path="/home/takosalad/llama/models/nous-capybara-3b-v1.9.q5_k_m.gguf",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/takosalad/.local/lib/python3.11/site-packages/llama_cpp/llama.py", line 305, in __init__
assert self.model is not None
^^^^^^^^^^^^^^^^^^^^^^
AssertionError
Exception ignored in:
Traceback (most recent call last):
File "/home/takosalad/.local/lib/python3.11/site-packages/llama_cpp/llama.py", line 1502, in __del__
if self.ctx is not None:
^^^^^^^^
AttributeError: 'Llama' object has no attribute 'ctx'
貢獻指南
評估
這個 Issue 還沒有評估資料。