abetlen / abetlen/llama-cpp-python

Allow python packages to contribute to LlamaChatCompletionHandlerRegistry

Aberta
#1,715 1 comentário 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
10.6k
Forks
1.4k
Métricas de merge de PRs
Métricas de PR pendentes

Descrição

**Is your feature request related to a problem? Please describe.**

Today, the [llama-cpp/llama_chat_format.py] contains 25 chat format, and 4 chat_completion_handler, this currently force the different actors to contribute to this never ending growing file.

This is the case for the [functionary](https://github.com/MeetKai/functionary) models, which has to keep updating the handlers to support their newer models.

- https://github.com/abetlen/llama-cpp-python/pull/784
- https://github.com/abetlen/llama-cpp-python/pull/1078
- https://github.com/abetlen/llama-cpp-python/pull/1282
- https://github.com/abetlen/llama-cpp-python/pull/1385
- https://github.com/abetlen/llama-cpp-python/pull/1419
- https://github.com/abetlen/llama-cpp-python/pull/1430
- https://github.com/abetlen/llama-cpp-python/pull/1509

This process can be slower than their pace of release since they have to get approval on this repository, the amazing people behind the functionary models have a repository with the necessary code to transform the generated content to proper CreateChatCompletionStreamResponse, and it would make sense that this would be their responsibility.

**Describe the solution you'd like**

python (>3.3) offers a lot of way to load code from other packages, or packages to contribute to a main packages. This would have a lot of advantages, as model provider could maintain their own packages, and rely on their own testing/versioning.

**Additional context**
Add any other context or screenshots about the feature request here.

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.