abetlen / abetlen/llama-cpp-python

Draft: Refactor Chat Formatter for Enhanced Flexibility and Extensibility

Aberta
#810 0 comentários 0 reações 0 responsáveis Ver no GitHub
enhancement
Linguagem predominante
Python
Estrelas
10.6k
Forks
1.4k
Métricas de merge de PRs
Métricas de PR pendentes

Descrição

**Is your feature request related to a problem? Please describe.**
The current chat formatter lacks flexibility and extensibility, especially when integrating new model templates. This results in cumbersome workarounds or "jerry-rigging" when trying to use models that don't have a supported template.

**Describe the solution you'd like**
I've introduced a refactoring in PR #809, which allows for a more flexible templating schema for different models. The changes allow users to easily create a new template and integrate it seamlessly. It also facilitates the process for contributors who want to add official support for their templates via PRs.

**Describe alternatives you've considered**
- Keeping the current architecture: This option is less appealing because it limits the growth and adaptability of the project, especially when introducing new models or adjusting to changing requirements.
- Overriding existing templates: While feasible, this approach is patchy and doesn't solve the root problem of inflexibility.

**Additional context**
The changes introduced are major and potentially breaking, but with careful integration and testing, they should enhance the codebase. This issue accompanies the Draft PR #809 to collect early feedback and ensure a smoother integration process. The objective is to make the tool more user-friendly and adaptive to a wider range of models and use-cases.

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.