abetlen / abetlen/llama-cpp-python

[query] Inputting Json to llama cpp and getting output into txt file ?

未關閉
#1,192 4 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
10.6k
分支
1.4k
PR 合併指標
PR 指標待擷取

描述

Hi I have a couple of json files that i want to input it to `llm `using `llama-cpp `. I am wondering if there is a way to do that . I am planning to use `mistral7b-instruct ` as a start.

The json file looks as follows
```
{
"title": "Some title ",
"description": "Some description",
"reference": "web_url",
"date": "xyz",
"popularity": 0,
"comments": [
{
"number": 1,
"content": "Some Text1"
},
{
"number": 2,
"content": "Some Text2"
}
]
}
```
I want to use llama cpp to pass
```
./main -m 'mistral mode ' - p "Describe briefly the problem {description} , summarize the comments in 10 bullet points {comments}
```
And output the llm into maybe a text with format

1. title
2. description
3. summary

Is it possible to do it via llama cpp or is it only possible through python bindings. i thought it would be much faster with llama cpp without the python bindings.

Has anyone tried something similar ?

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。