vllm-project / vllm-project/agentic-api
feat: Add Tavily as a selectable web_search provider
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 284
- Forks
- 74
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 93
Description
Problem statement / motivation
Part of #291
Status: Ready for implementation (builds on typed provider contract established in #293 and #294)
Scope: Tavily provider integration, configuration plumbing, and tests
Tavily is a premier search engine purpose-built for AI agents and LLM applications (used by LangChain, AutoGen, and CrewAI):
- Agent-optimized content: Cleans webpage content, filters out SEO spam/clickbait, and provides high-density, factual snippets directly optimized for LLM context windows.
- Native domain filtering: Native server-side support for
include_domainsandexclude_domains. - News and raw content support: Direct support for news topic filtering and answer synthesis.
- Differences from You.com / Brave: Tavily uses
POST /searchwith a JSON payload (rather thanGETwith query parameters), authenticates viaapi_keyin body orAuthorization: Bearerheader, and structures results withmax_results(up to 20).
Proposed solution
A. Provider Implementation (tool/web_search/tavily.rs)
- Endpoint:
POST {base_url}/searchwithContent-Type: application/json. - Authentication:
Authorization: Bearer {api_key}header (orapi_keyin payload). Key resolved fromTAVILY_API_KEY. - Deserialization: Forward-compatible structs with
#[serde(default)]mapping Tavily's response envelope (results: [{ title, url, content, published_date }]) toWebSearchProviderResponse. - Request body payload:
{
"query": "...",
"search_depth": "basic",
"max_results": 10,
"topic": "general",
"include_domains": [],
"exclude_domains": []
}
B. Input Adaptation Policy Matrix
| Parameter / Input | Tavily Behavior | Gateway Adaptation Policy |
|---|---|---|
allowed_domains / blocked_domains |
Native server-side support (include_domains / exclude_domains) |
Forwarded directly in JSON body; fallback client-side DomainFilter as a defense-in-depth guarantee. |
count |
max_results (1..=20) |
Mapped to max_results; clamp to 20 with tracing::debug!. |
freshness |
days parameter for recency (1, 7, 30) or time_range |
Map Freshness::Day -> days: 1, Week -> days: 7, Month -> days: 30, or best-effort date range mapping. |
| News | topic: "news" |
When news is requested or present, set topic: "news". |
You.com specifics (livecrawl, etc.) |
Tavily has its own include_raw_content option |
Ignored for base tool compatibility with tracing::debug!. |
C. Concurrency & Rate Limiting
- Concurrency Ceiling: Configurable via
AGENTIC_WEB_SEARCH_MAX_CONCURRENT_QUERIES, defaulting to gateway limit or tier allowance. - 429 Handling: Fail
web_search_callwithout retry, surfacingRetry-Afterverbatim per RFC 7231. - Auth Errors (401/403): Report failure explicitly naming
TAVILY_API_KEYwithout leaking credentials or echoing response body.
D. Configuration & Precedence Matrix
| Setting | Environment Variable | config.toml Key |
Default Value |
|---|---|---|---|
| Provider | AGENTIC_WEB_SEARCH_PROVIDER |
[web_search].provider |
"you" |
| API Key | Named by api_key_env |
[web_search].api_key_env |
"TAVILY_API_KEY" (for Tavily) |
| Base URL | AGENTIC_WEB_SEARCH_BASE_URL |
[web_search].base_url |
https://api.tavily.com |
| Max Concurrency | AGENTIC_WEB_SEARCH_MAX_CONCURRENT_QUERIES |
[web_search].max_concurrent_queries |
Inherits gateway limit |
Scope Boundaries
| In Scope | Explicitly Out of Scope (Deferred) |
|---|---|
✅ TavilySearchProvider implementation (POST /search) |
❌ include_raw_content / large body scraping payloads |
| ✅ Native domain allow/block list forwarding | ❌ Tavily Q&A answer injection (include_answer) |
| ✅ News topic adaptation & count clamping | ❌ Webhook / asynchronous search polling |
✅ Axum mock HTTP tests (tests/web_search_tavily_test.rs) |
❌ Custom reranking scoring thresholds |
Verification Checklist
- 200 OK (Search & News): Verify JSON
POSTpayload shapes correctly and responses map toWebSearchResult. - Domain Filtering: Verify native
include_domains/exclude_domainsJSON fields are populated. - 401 / 403 Authentication: Fails
web_search_call, referencesTAVILY_API_KEY, does not leak credentials. - 429 Rate Limit: Surfaces
Retry-Afterverbatim without retrying. - Metadata: Echoes
"provider": "tavily"inmetadata[]. - Zero Regressions: You.com and Brave Search remain byte-identical and unaffected.
- Hygiene:
cargo fmt,cargo clippy --all-targets -- -D warnings, and file size limits pass.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the typed provider contract established in issues #293 and #294, then inspect tool/web_search/tavily.rs and the existing provider implementations. Run or extend tests/web_search_tavily_test.rs using Axum mock HTTP tests. Done means the documented request mappings, authentication and error handling, metadata, domain filtering, and regression checks pass with cargo fmt and clippy.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, backend, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 70/100