abetlen / abetlen/llama-cpp-python

Draft: Refactor Chat Formatter for Enhanced Flexibility and Extensibility

Abierto
#810 0 comentarios 0 reacciones 0 asignados Ver en GitHub
enhancement
Lenguaje dominante
Python
Estrellas
10.6k
Forks
1.4k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

**Is your feature request related to a problem? Please describe.**
The current chat formatter lacks flexibility and extensibility, especially when integrating new model templates. This results in cumbersome workarounds or "jerry-rigging" when trying to use models that don't have a supported template.

**Describe the solution you'd like**
I've introduced a refactoring in PR #809, which allows for a more flexible templating schema for different models. The changes allow users to easily create a new template and integrate it seamlessly. It also facilitates the process for contributors who want to add official support for their templates via PRs.

**Describe alternatives you've considered**
- Keeping the current architecture: This option is less appealing because it limits the growth and adaptability of the project, especially when introducing new models or adjusting to changing requirements.
- Overriding existing templates: While feasible, this approach is patchy and doesn't solve the root problem of inflexibility.

**Additional context**
The changes introduced are major and potentially breaking, but with careful integration and testing, they should enhance the codebase. This issue accompanies the Draft PR #809 to collect early feedback and ensure a smoother integration process. The objective is to make the tool more user-friendly and adaptive to a wider range of models and use-cases.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.