abetlen / abetlen/llama-cpp-python

System role not supported Gemma 2

オープン
#1,580 コメント 5 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Python
スター
10.6k
フォーク
1.4k
PR マージ指標
PR 指標を取得中

説明

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [ ] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [ ] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [ X] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [X ] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.

# Expected Behavior

I tried Gemma2 GGUF from hugging face. It is working fine with llama.cpp CLI on my PC.
However , I got issues while runing through llama-cpp-python package as follow :

self.llama = Llama(model_path=model_path, n_gpu_layers=n_gpu_layers, n_ctx=n_ctx, echo=echo, n_batch=n_batch,flash_attn=True)

messages = [
{
"role": "system",
"content": "You are a helpful assistant who perfectly reply to user request on the topic of Mistrious Dragon"
},
{
"role": "user",
"content": "Write a beautiful stroy for kids"
}
]

output = self.llama.create_chat_completion(messages,
temperature=temperature,
max_tokens=max_tokens,
top_p=1,
frequency_penalty=1.5,
presence_penalty=1.5)

# Current Behavior

I got the following error:
......................python3.11/site-packages/llama_cpp/llama_chat_format.py", line 213, in raise_exception
raise ValueError(message)
ValueError: System role not supported

# Environment and Context

I use python3.11 on ubuntu linux with CUDA enabled . > 24GB GPU RAM

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。