abetlen / abetlen/llama-cpp-python

Feat: Add support for Llama 3.1 function calling

Ouverte
#1,618 3 commentaires 21 réactions 0 personnes assignées Voir sur GitHub
enhancement
Langage dominant
Python
Étoiles
10.6k
Forks
1.4k
Métriques de merge des PR
Métriques de PR en attente

Description

Llama 3.1 models now officially supports function calling. So I think it would be nice to have function calling support for Llama 3.1 alongside existing functionary models.

Looks like we have more roles now!

[system, user, assistant, ipython] ; the last one being used to mark messages with the output of a tool's call!

The model calls its `built in` tools with the token `<|python_tag|>` and user defined tools as instructed in system prompt ends its turn with either the `<|eot_id|>` token for user defined tools or the `<|eom_id|>` token for built in tools.

The built in tools are "brave search, wolfram_alpha and code interpreter.

More information can be found here: https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_1/

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.