abetlen / abetlen/llama-cpp-python

Help needed with loading 'TheBloke/Mistral-7B-Instruct-v0.1-GGUF' model using `llama-cpp-python`

Abierto
#915 8 comentarios 0 reacciones 0 asignados Ver en GitHub
question
Lenguaje dominante
Python
Estrellas
10.6k
Forks
1.4k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

Hello,

I'm currently working on a project that requires the use of the `TheBloke/Mistral-7B-Instruct-v0.1-GGUF` model, which is in the GGUF format. I've tried using the Hugging Face library to load this model, but it seems that the library does not support the GGUF format.

I've also tried using the `ctransformers` library, but I've encountered some issues with it as well. Therefore, I'm considering using the llama-cpp-python library instead.

However, I'm having trouble understanding how to use the llama-cpp-python library to load the 'TheBloke/Mistral-7B-Instruct-v0.1-GGUF' model. The documentation for the llama-cpp-python library is not very detailed, and there are no specific examples of how to use this library to load a model from the Hugging Face Model Hub.

I would greatly appreciate if you could provide some guidance on how to use the llama-cpp-python library to load the `TheBloke/Mistral-7B-Instruct-v0.1-GGUF` model. Specifically, I would like to know how to install the library, how to import it in my Python code, and how to use it to load the model.

Thank you in advance for your help.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.