abetlen / abetlen/llama-cpp-python
Models with multiple chat templates
Open
enhancement
- Dominant language
- Python
- Stars
- 10.6k
- Forks
- 1.4k
- PR merge metrics
- PR metrics pending
Description
Not an issue yet, but will need to be handled once [this](https://github.com/ggerganov/llama.cpp/issues/6484) is implemented based on recent transformers [changes](https://github.com/huggingface/transformers/pull/29658).
Also note the kwargs change in the same PR which will be used by f.ex. C4AI Command R models (new chat template is not merged yet) to pass along `tools` and `documents`, while we already support `tools`, it might be worthwhile to support other things.
Contributor guide
Assessment
This issue has not been assessed yet.