abetlen / abetlen/llama-cpp-python

Add support T5 (encode-decoder) models at API level and server

Open
#1,681 0 comments 9 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
10.6k
Forks
1.4k
PR merge metrics
PR metrics pending

Description

Ciao Andrei,
following the latest changes on llama.cpp repo
[https://github.com/ggerganov/llama.cpp/issues/5763](https://github.com/ggerganov/llama.cpp/issues/5763)

can you tell me if this is going to be art of your amazing repo?
In case it is not, can you provide some instructions to replicate at low Level API the same steps done by lama-cli?

we can start using amazing models like LaMini-Flan-T5-248M with llama-cpp-python.

thanks

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.