abetlen / abetlen/llama-cpp-python

I'm loading a gguf but llama-python-cpp says "is it really a GGML file" ???

未關閉
#1,191 1 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
10.6k
分支
1.4k
PR 合併指標
PR 指標待擷取

描述

Installed llama_cpp_python-0.2.43.tar.gz via
CMAKE_ARGS="-DLLAMA_CUBLAS=ON -DCMAKE_CUDA_COMPILER=/opt/cuda/bin/nvcc -DTCNN_CUDA_ARCHITECTURES=61" pip install llama-cpp-python

llm = Llama(model_path="/home/takosalad/llama/models/nous-capybara-3b-v1.9.q5_k_m.gguf",
n_gpu_layers=63, n_threads=6, n_ctx=3584, n_batch=512, verbose=True),

ggml_init_cublas: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9
llama.cpp: loading model from /home/takosalad/llama/models/nous-capybara-3b-v1.9.q5_k_m.gguf
error loading model: unknown (magic, version) combination: 46554747, 00000003; is this really a GGML file?
llama_load_model_from_file: failed to load model
Traceback (most recent call last):
File "/home/takosalad/llama/./test.py", line 35, in
llm = Llama(model_path="/home/takosalad/llama/models/nous-capybara-3b-v1.9.q5_k_m.gguf",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/takosalad/.local/lib/python3.11/site-packages/llama_cpp/llama.py", line 305, in __init__
assert self.model is not None
^^^^^^^^^^^^^^^^^^^^^^
AssertionError
Exception ignored in:
Traceback (most recent call last):
File "/home/takosalad/.local/lib/python3.11/site-packages/llama_cpp/llama.py", line 1502, in __del__
if self.ctx is not None:
^^^^^^^^
AttributeError: 'Llama' object has no attribute 'ctx'

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。