Per-agent option to exclude built-in knowledge-base search action (MCP-only agents); enable_kb=false does not remove it when KB is configured globally

Summary

When the knowledge base is configured globally (agent pool / DATABASE_URL, VECTOR_ENGINE=postgres), the built-in KB search action (search_memory in main; surfaced as search_collection in our LocalAI build) is added to every agent's tool set. There is no per-agent way to exclude it. Setting the per-agent flags enable_kb=false, kb_as_tools=false, kb_auto_search=false does not remove the action from the agent's available tools.

As a result, an agent that is meant to use a specific set of MCP tools can have its tool selection hijacked: the tool-selection layer (cogito pick_tool) prefers the built-in KB search over the intended MCP tool.

Environment

• LocalAGI embedded in LocalAI (agent pool), KB backend = postgres (LocalRecall), LOCALAI_AGENT_POOL_DATABASE_URL set globally.
• Agent mcp_servers = a few narrow MCP groups (no KB/search tool among them).
• Agent flags: enable_kb=false, kb_as_tools=false, kb_auto_search=false.

Steps to reproduce

1. Configure a global KB (postgres/agent-pool DB) with at least one populated collection (e.g. documents).
2. Create an agent whose mcp_servers expose a document-by-id tool (e.g. an MCP get_document_content(id)), and set enable_kb=false, kb_as_tools=false.
3. Run a task: “Fetch the content of document ID 23442 via get_document_content, then …”.

Expected

With enable_kb=false, the agent should NOT be offered the built-in KB search action; it should use the MCP get_document_content.

Actual

• The built-in KB search action is still present in the agent's tool set.
• pick_tool selects the KB search (semantic search over the collection) instead of the MCP get_document_content.
• The agent searches for the document (query = "get_document_content" / the id) → 0 results → fails the task, never calling the MCP tool.
• Confirmed the MCP tools do load (hub logs show initialize + tools/list), and the KB action is genuinely the built-in (it returns real reranked collection results), not an MCP tool and not a hallucination.

Impact

Any deployment that runs both (a) a global KB and (b) MCP-only specialist agents cannot reliably build the specialist agent — the KB search competes with and overrides the MCP tools. There is no documented workaround.

Proposed solutions (any one)

1. Make enable_kb=false (or a new disable_builtin_kb_tools / kb_as_tools=false) actually remove the KB search/add actions from the agent's available tools.
2. Add a per-agent tool allow-list / deny-list (e.g. allowed_tools, excluded_tools) so an operator can constrain the agent to exactly the MCP tools.
3. Improve pick_tool (cogito) so an explicit system-prompt instruction / enable_guided_tools reliably steers selection toward a named tool over the built-in KB search.

Workarounds tried (all ineffective)

Recreated the agent and re-ran the same task with each of these — the built-in KB search action
remained available and pick_tool kept choosing it over the MCP tool:

• enable_kb=false
• kb_as_tools=false
• kb_auto_search=false
• enable_guided_tools=true
• local_rag_url="" (empty)

There appears to be no per-agent configuration that yields an MCP-tools-only agent when a global
KB is configured. This is the core of the request.

Notes

• Tool-selection logic lives in mudler/cogito (pick_tool/intention); enable_guided_tools maps to its guideline system.
• In current main the built-in action is search_memory/add_memory (core/agent/knowledgebase.go), gated by enableKB for the auto-search path (knowledgeBaseLookup), but the action availability appears independent of the per-agent flag when the KB is globally wired.

Start in core/agent/knowledgebase.go, tracing how globally configured knowledge-base actions are added to an agent and how enableKB affects availability; then inspect mudler/cogito’s pick_tool/intention path. Verify the behavior with an MCP document tool and a globally configured KB, and consider the issue done when the disabled built-in action is absent and the MCP tool is selected.