abetlen / abetlen/llama-cpp-python

Feat: Add support for Llama 3.1 function calling

Abierto
#1,618 3 comentarios 21 reacciones 0 asignados Ver en GitHub
enhancement
Lenguaje dominante
Python
Estrellas
10.6k
Forks
1.4k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

Llama 3.1 models now officially supports function calling. So I think it would be nice to have function calling support for Llama 3.1 alongside existing functionary models.

Looks like we have more roles now!

[system, user, assistant, ipython] ; the last one being used to mark messages with the output of a tool's call!

The model calls its `built in` tools with the token `<|python_tag|>` and user defined tools as instructed in system prompt ends its turn with either the `<|eot_id|>` token for user defined tools or the `<|eom_id|>` token for built in tools.

The built in tools are "brave search, wolfram_alpha and code interpreter.

More information can be found here: https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_1/

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.