SciSharp / SciSharp/BotSharp

KnowledgeBase doesn't work with Llama-Sharp on local LLM

Open
#1,086 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C#
Stars
3.1k
Forks
651
Avg merge
15h 2m
Merged PRs (30d)
36

Description

I have configured Llama-Sharp and I am using the llama3.1-8b-instruct-q6_k.gguf model on my cpu. I have installed the knowledgebase plugin and I have also installed and configured qdrant. The knowledge that I have installed in qdrant is of the Q&A type. When I ask a question in the chat, Llama Sharp does not use knowledge and gives the user the answer through LLM. I have put the agent configuration below.
{
"id": "121a6dab-afa9-469f-bbe6-8df111b77700",
"name": "Q&A bot",
"description": "This is a Q&A bot",
"type": "task",
"mode": "eager",
"createdDateTime": "2025-05-10T12:45:57.3500389Z",
"updatedDateTime": "2025-07-02T09:52:29.17704Z",
"llmConfig": {
"is_inherit": false,
"provider": "llama-sharp",
"model": "llama3.1-8b-instruct-q6_k.gguf",
"max_recursion_depth": 16,
"max_output_tokens": 4096
},
"isPublic": true,
"disabled": false,
"iconUrl": null,
"profiles": [],
"labels": [],
"mergeUtility": false,
"utilities": [
{
"name": "kg.knowledge-base",
"disabled": false,
"functions": [
{
"name": "util-kg-knowledge_retrieval"
}
],
"templates": [
{
"name": "util-kg-knowledge_retrieval.fn"
}
]
}
],
"mcpTools": [],
"rules": [],
"knowledgeBases": [
{
"name": "BotSharp",
"type": "question-answer",
"disabled": false,
"confidence": 0.1
}
],
"inheritAgentId": null,
"maxMessageCount": 4096,
"routingRules": []
}

And the Instruct was as follows:
"
You are a helpful Q&A assistant. When the user asks a question, search the knowledge base using the 'util-kg-knowledge_retrieval' function to retrieve relevant information, then respond clearly and helpfully based on the retrieved knowledge. If no relevant knowledge is found, politely inform the user. Always keep answers accurate and easy to understand.

Use this function for retrieving knowledge "util-kg-knowledge_retrieval"
"

I ask the experts to help me solve this problem.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the configured Q&A agent with Llama-Sharp, Qdrant, and the llama3.1-8b-instruct-q6_k.gguf model, then trace the util-kg-knowledge_retrieval function named in the configuration. Done means a question invokes knowledge retrieval and the response uses retrieved knowledge, or clearly reports when none is found.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
ai, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.