abetlen / abetlen/llama-cpp-python

[query] Inputting Json to llama cpp and getting output into txt file ?

Aberta
#1,192 4 comentários 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
10.6k
Forks
1.4k
Métricas de merge de PRs
Métricas de PR pendentes

Descrição

Hi I have a couple of json files that i want to input it to `llm `using `llama-cpp `. I am wondering if there is a way to do that . I am planning to use `mistral7b-instruct ` as a start.

The json file looks as follows
```
{
"title": "Some title ",
"description": "Some description",
"reference": "web_url",
"date": "xyz",
"popularity": 0,
"comments": [
{
"number": 1,
"content": "Some Text1"
},
{
"number": 2,
"content": "Some Text2"
}
]
}
```
I want to use llama cpp to pass
```
./main -m 'mistral mode ' - p "Describe briefly the problem {description} , summarize the comments in 10 bullet points {comments}
```
And output the llm into maybe a text with format

1. title
2. description
3. summary

Is it possible to do it via llama cpp or is it only possible through python bindings. i thought it would be much faster with llama cpp without the python bindings.

Has anyone tried something similar ?

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.