abetlen / abetlen/llama-cpp-python

I'm loading a gguf but llama-python-cpp says "is it really a GGML file" ???

Aperta
#1,191 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
10.6k
Fork
1.4k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

Installed llama_cpp_python-0.2.43.tar.gz via
CMAKE_ARGS="-DLLAMA_CUBLAS=ON -DCMAKE_CUDA_COMPILER=/opt/cuda/bin/nvcc -DTCNN_CUDA_ARCHITECTURES=61" pip install llama-cpp-python

llm = Llama(model_path="/home/takosalad/llama/models/nous-capybara-3b-v1.9.q5_k_m.gguf",
n_gpu_layers=63, n_threads=6, n_ctx=3584, n_batch=512, verbose=True),

ggml_init_cublas: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9
llama.cpp: loading model from /home/takosalad/llama/models/nous-capybara-3b-v1.9.q5_k_m.gguf
error loading model: unknown (magic, version) combination: 46554747, 00000003; is this really a GGML file?
llama_load_model_from_file: failed to load model
Traceback (most recent call last):
File "/home/takosalad/llama/./test.py", line 35, in
llm = Llama(model_path="/home/takosalad/llama/models/nous-capybara-3b-v1.9.q5_k_m.gguf",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/takosalad/.local/lib/python3.11/site-packages/llama_cpp/llama.py", line 305, in __init__
assert self.model is not None
^^^^^^^^^^^^^^^^^^^^^^
AssertionError
Exception ignored in:
Traceback (most recent call last):
File "/home/takosalad/.local/lib/python3.11/site-packages/llama_cpp/llama.py", line 1502, in __del__
if self.ctx is not None:
^^^^^^^^
AttributeError: 'Llama' object has no attribute 'ctx'

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.