abetlen / abetlen/llama-cpp-python

Draft: Refactor Chat Formatter for Enhanced Flexibility and Extensibility

未關閉
#810 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
enhancement
主要語言
Python
星號
10.6k
分支
1.4k
PR 合併指標
PR 指標待擷取

描述

**Is your feature request related to a problem? Please describe.**
The current chat formatter lacks flexibility and extensibility, especially when integrating new model templates. This results in cumbersome workarounds or "jerry-rigging" when trying to use models that don't have a supported template.

**Describe the solution you'd like**
I've introduced a refactoring in PR #809, which allows for a more flexible templating schema for different models. The changes allow users to easily create a new template and integrate it seamlessly. It also facilitates the process for contributors who want to add official support for their templates via PRs.

**Describe alternatives you've considered**
- Keeping the current architecture: This option is less appealing because it limits the growth and adaptability of the project, especially when introducing new models or adjusting to changing requirements.
- Overriding existing templates: While feasible, this approach is patchy and doesn't solve the root problem of inflexibility.

**Additional context**
The changes introduced are major and potentially breaking, but with careful integration and testing, they should enhance the codebase. This issue accompanies the Draft PR #809 to collect early feedback and ensure a smoother integration process. The objective is to make the tool more user-friendly and adaptive to a wider range of models and use-cases.

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。