abetlen / abetlen/llama-cpp-python

Allow python packages to contribute to LlamaChatCompletionHandlerRegistry

Đang mở
#1,715 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
10.6k
Fork
1.4k
Chỉ số merge pull request
Chỉ số pull request đang chờ

Mô tả

**Is your feature request related to a problem? Please describe.**

Today, the [llama-cpp/llama_chat_format.py] contains 25 chat format, and 4 chat_completion_handler, this currently force the different actors to contribute to this never ending growing file.

This is the case for the [functionary](https://github.com/MeetKai/functionary) models, which has to keep updating the handlers to support their newer models.

- https://github.com/abetlen/llama-cpp-python/pull/784
- https://github.com/abetlen/llama-cpp-python/pull/1078
- https://github.com/abetlen/llama-cpp-python/pull/1282
- https://github.com/abetlen/llama-cpp-python/pull/1385
- https://github.com/abetlen/llama-cpp-python/pull/1419
- https://github.com/abetlen/llama-cpp-python/pull/1430
- https://github.com/abetlen/llama-cpp-python/pull/1509

This process can be slower than their pace of release since they have to get approval on this repository, the amazing people behind the functionary models have a repository with the necessary code to transform the generated content to proper CreateChatCompletionStreamResponse, and it would make sense that this would be their responsibility.

**Describe the solution you'd like**

python (>3.3) offers a lot of way to load code from other packages, or packages to contribute to a main packages. This would have a lot of advantages, as model provider could maintain their own packages, and rely on their own testing/versioning.

**Additional context**
Add any other context or screenshots about the feature request here.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.