abetlen / abetlen/llama-cpp-python

Allow python packages to contribute to LlamaChatCompletionHandlerRegistry

Aperta
#1,715 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
10.6k
Fork
1.4k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

**Is your feature request related to a problem? Please describe.**

Today, the [llama-cpp/llama_chat_format.py] contains 25 chat format, and 4 chat_completion_handler, this currently force the different actors to contribute to this never ending growing file.

This is the case for the [functionary](https://github.com/MeetKai/functionary) models, which has to keep updating the handlers to support their newer models.

- https://github.com/abetlen/llama-cpp-python/pull/784
- https://github.com/abetlen/llama-cpp-python/pull/1078
- https://github.com/abetlen/llama-cpp-python/pull/1282
- https://github.com/abetlen/llama-cpp-python/pull/1385
- https://github.com/abetlen/llama-cpp-python/pull/1419
- https://github.com/abetlen/llama-cpp-python/pull/1430
- https://github.com/abetlen/llama-cpp-python/pull/1509

This process can be slower than their pace of release since they have to get approval on this repository, the amazing people behind the functionary models have a repository with the necessary code to transform the generated content to proper CreateChatCompletionStreamResponse, and it would make sense that this would be their responsibility.

**Describe the solution you'd like**

python (>3.3) offers a lot of way to load code from other packages, or packages to contribute to a main packages. This would have a lot of advantages, as model provider could maintain their own packages, and rely on their own testing/versioning.

**Additional context**
Add any other context or screenshots about the feature request here.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.