abetlen / abetlen/llama-cpp-python

phi3 chat format

Open
#1,736 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10.6k
Forks
1.4k
PR merge metrics
PR metrics pending

Description

Hello,

Could you add the chat format for the phi3 models ?

It is described [here](https://huggingface.co/microsoft/Phi-3-medium-128k-instruct) :
```bash
<|user|>\nQuestion<|end|>\n<|assistant|>\nAnswer<|end|>\n
```

Maybe something like this in `llama_chat_format.py` ?

```python
@register_chat_format("phi3")
def format_phi3(
messages: List[llama_types.ChatCompletionRequestMessage],
**kwargs: Any,
) -> ChatFormatterResponse:
_roles = dict(
user="<|user|>\n",
assistant="<|assistant|>\n",
)
_sep = "<|end|>\n"
_messages = _map_roles(messages, _roles)
_messages.append((_roles["assistant"], None))
_prompt = _format_no_colon_single("", _messages, _sep)
return ChatFormatterResponse(prompt=_prompt, stop=_sep)
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.