agentscope-ai / agentscope-ai/QwenPaw

[Feature]: Add SearXNG support

Ouverte
#3,004 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub
enhancement
Langage dominant
Python
Étoiles
34.9k
Forks
3.1k
Merge moyen
1 j 15 h
PR mergées (30 j)
225

Description

## Summary
Add a web_search built-in tool with support for SearXNG as a pluggable search backend, enabling privacy-friendly and self-hostable web search capabilities in Copaw.
## Component(s) Affected

- [ ✅] Core / Backend (app, agents, config, providers, utils, local_models)
- [ ] Console (frontend web UI)
- [ ] Channels (DingTalk, Feishu, QQ, Discord, iMessage, etc.)
- [ ] Skills
- [ ] CLI
- [ ] Documentation (website)
- [ ] Tests
- [ ] CI/CD
- [ ] Scripts / Deploy

## Problem / Motivation

Currently, Copaw lacks a unified and extensible web_search tool in its built-in toolset.

This leads to several issues:

Agents cannot easily access real-time or external knowledge
Users must rely on external integrations or custom tools
No privacy-friendly alternative to commercial search APIs (e.g., Google, Bing)
Difficult to standardize search behavior across agents

Who benefits:

Developers building agents that require up-to-date information
Self-hosting users who prefer privacy and control over search infrastructure
Teams that want a unified tool interface for retrieval

## Proposed Solution

Introduce a built-in web_search tool with a pluggable provider architecture, starting with support for SearXNG.

1. Tool Interface

Define a standard tool schema:

web_search_tool = {
"name": "web_search",
"description": "Search the web for up-to-date information",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string"},
"top_k": {"type": "integer", "default": 5}
},
"required": ["query"]
}
}
2. Provider Abstraction

Define a base search provider interface:

from abc import ABC, abstractmethod
from typing import List, Optional, Dict

class SearchResult(dict):
title: str
url: str
snippet: str
source: Optional[str]

class SearchProvider(ABC):

@abstractmethod
def search(self, query: str, top_k: int = 5) -> List[Dict]:
pass
3. SearXNG Provider

Implement a SearXNG-based provider:

import requests
from typing import List, Dict

class SearxngSearchProvider(SearchProvider):

def __init__(self, base_url: str, timeout: int = 5):
self.base_url = base_url.rstrip("/")
self.timeout = timeout

def search(self, query: str, top_k: int = 5) -> List[Dict]:
url = f"{self.base_url}/search"
params = {
"q": query,
"format": "json"
}

resp = requests.get(url, params=params, timeout=self.timeout)
resp.raise_for_status()

data = resp.json()
results = data.get("results", [])

parsed = []
for item in results[:top_k]:
parsed.append({
"title": item.get("title"),
"url": item.get("url"),
"snippet": item.get("content"),
"source": item.get("engine")
})

return parsed
4. Configuration

Add configuration support:

web_search:
provider: searxng
searxng:
base_url: http://localhost:8080
timeout: 5
5. Tool Wrapper / Execution Layer

Wrap provider into a callable tool:

class WebSearchTool:

def __init__(self, provider: SearchProvider):
self.provider = provider

def run(self, query: str, top_k: int = 5):
return self.provider.search(query=query, top_k=top_k)
6. Agent Integration
Register web_search as a built-in tool
Allow agents to invoke it via tool calling
Optionally add prompting strategy:
Use web_search when:
- The query requires real-time information
- The answer is not in model knowledge
- External references are needed
7. Extensibility

Design allows future providers:

Bing Search API
Google Programmable Search
DuckDuckGo
Local hybrid search (RAG + web)

## Alternatives Considered

Direct integration with commercial APIs
→ Rejected due to API cost, rate limits, and lack of privacy
Custom user-defined tools only
→ Too fragmented; lacks standardization
RAG-only approach (no web search)
→ Cannot handle real-time or unknown queries

## Additional Context

Openclaw officially supported the feature:https://docs.openclaw.ai/tools/searxng-search

## Willing to Contribute

- [ ✅] I am willing to open a PR for this feature (after discussion).

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.