abetlen / abetlen/llama-cpp-python

Feat: Add support for Llama 3.1 function calling

Aberta
#1,618 3 comentários 21 reações 0 responsáveis Ver no GitHub
enhancement
Linguagem predominante
Python
Estrelas
10.6k
Forks
1.4k
Métricas de merge de PRs
Métricas de PR pendentes

Descrição

Llama 3.1 models now officially supports function calling. So I think it would be nice to have function calling support for Llama 3.1 alongside existing functionary models.

Looks like we have more roles now!

[system, user, assistant, ipython] ; the last one being used to mark messages with the output of a tool's call!

The model calls its `built in` tools with the token `<|python_tag|>` and user defined tools as instructed in system prompt ends its turn with either the `<|eot_id|>` token for user defined tools or the `<|eom_id|>` token for built in tools.

The built in tools are "brave search, wolfram_alpha and code interpreter.

More information can be found here: https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_1/

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.