abetlen / abetlen/llama-cpp-python

Help needed with loading 'TheBloke/Mistral-7B-Instruct-v0.1-GGUF' model using `llama-cpp-python`

Offen
#915 8 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
question
Vorherrschende Sprache
Python
Sterne
10.6k
Forks
1.4k
PR-Merge-Kennzahlen
PR-Kennzahlen ausstehend

Beschreibung

Hello,

I'm currently working on a project that requires the use of the `TheBloke/Mistral-7B-Instruct-v0.1-GGUF` model, which is in the GGUF format. I've tried using the Hugging Face library to load this model, but it seems that the library does not support the GGUF format.

I've also tried using the `ctransformers` library, but I've encountered some issues with it as well. Therefore, I'm considering using the llama-cpp-python library instead.

However, I'm having trouble understanding how to use the llama-cpp-python library to load the 'TheBloke/Mistral-7B-Instruct-v0.1-GGUF' model. The documentation for the llama-cpp-python library is not very detailed, and there are no specific examples of how to use this library to load a model from the Hugging Face Model Hub.

I would greatly appreciate if you could provide some guidance on how to use the llama-cpp-python library to load the `TheBloke/Mistral-7B-Instruct-v0.1-GGUF` model. Specifically, I would like to know how to install the library, how to import it in my Python code, and how to use it to load the model.

Thank you in advance for your help.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.