abetlen / abetlen/llama-cpp-python

I'm loading a gguf but llama-python-cpp says "is it really a GGML file" ???

Aberta
#1,191 1 comentário 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
10.6k
Forks
1.4k
Métricas de merge de PRs
Métricas de PR pendentes

Descrição

Installed llama_cpp_python-0.2.43.tar.gz via
CMAKE_ARGS="-DLLAMA_CUBLAS=ON -DCMAKE_CUDA_COMPILER=/opt/cuda/bin/nvcc -DTCNN_CUDA_ARCHITECTURES=61" pip install llama-cpp-python

llm = Llama(model_path="/home/takosalad/llama/models/nous-capybara-3b-v1.9.q5_k_m.gguf",
n_gpu_layers=63, n_threads=6, n_ctx=3584, n_batch=512, verbose=True),

ggml_init_cublas: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9
llama.cpp: loading model from /home/takosalad/llama/models/nous-capybara-3b-v1.9.q5_k_m.gguf
error loading model: unknown (magic, version) combination: 46554747, 00000003; is this really a GGML file?
llama_load_model_from_file: failed to load model
Traceback (most recent call last):
File "/home/takosalad/llama/./test.py", line 35, in
llm = Llama(model_path="/home/takosalad/llama/models/nous-capybara-3b-v1.9.q5_k_m.gguf",
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/takosalad/.local/lib/python3.11/site-packages/llama_cpp/llama.py", line 305, in __init__
assert self.model is not None
^^^^^^^^^^^^^^^^^^^^^^
AssertionError
Exception ignored in:
Traceback (most recent call last):
File "/home/takosalad/.local/lib/python3.11/site-packages/llama_cpp/llama.py", line 1502, in __del__
if self.ctx is not None:
^^^^^^^^
AttributeError: 'Llama' object has no attribute 'ctx'

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.