abetlen / abetlen/llama-cpp-python

Help needed with loading 'TheBloke/Mistral-7B-Instruct-v0.1-GGUF' model using `llama-cpp-python`

オープン
#915 コメント 8 件 リアクション 0 件 担当者 0 名 GitHub で見る
question
主要言語
Python
スター
10.6k
フォーク
1.4k
PR マージ指標
PR 指標を取得中

説明

Hello,

I'm currently working on a project that requires the use of the `TheBloke/Mistral-7B-Instruct-v0.1-GGUF` model, which is in the GGUF format. I've tried using the Hugging Face library to load this model, but it seems that the library does not support the GGUF format.

I've also tried using the `ctransformers` library, but I've encountered some issues with it as well. Therefore, I'm considering using the llama-cpp-python library instead.

However, I'm having trouble understanding how to use the llama-cpp-python library to load the 'TheBloke/Mistral-7B-Instruct-v0.1-GGUF' model. The documentation for the llama-cpp-python library is not very detailed, and there are no specific examples of how to use this library to load a model from the Hugging Face Model Hub.

I would greatly appreciate if you could provide some guidance on how to use the llama-cpp-python library to load the `TheBloke/Mistral-7B-Instruct-v0.1-GGUF` model. Specifically, I would like to know how to install the library, how to import it in my Python code, and how to use it to load the model.

Thank you in advance for your help.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。