abetlen / abetlen/llama-cpp-python

Draft: Refactor Chat Formatter for Enhanced Flexibility and Extensibility

Ouverte
#810 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
enhancement
Langage dominant
Python
Étoiles
10.6k
Forks
1.4k
Métriques de merge des PR
Métriques de PR en attente

Description

**Is your feature request related to a problem? Please describe.**
The current chat formatter lacks flexibility and extensibility, especially when integrating new model templates. This results in cumbersome workarounds or "jerry-rigging" when trying to use models that don't have a supported template.

**Describe the solution you'd like**
I've introduced a refactoring in PR #809, which allows for a more flexible templating schema for different models. The changes allow users to easily create a new template and integrate it seamlessly. It also facilitates the process for contributors who want to add official support for their templates via PRs.

**Describe alternatives you've considered**
- Keeping the current architecture: This option is less appealing because it limits the growth and adaptability of the project, especially when introducing new models or adjusting to changing requirements.
- Overriding existing templates: While feasible, this approach is patchy and doesn't solve the root problem of inflexibility.

**Additional context**
The changes introduced are major and potentially breaking, but with careful integration and testing, they should enhance the codebase. This issue accompanies the Draft PR #809 to collect early feedback and ensure a smoother integration process. The objective is to make the tool more user-friendly and adaptive to a wider range of models and use-cases.

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.