vllm-project / vllm-project/agentic-api

feat: Add Tavily as a selectable web_search provider

Open
#327 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Rust
Stars
284
Forks
74
Avg merge
1d 17h
Merged PRs (30d)
93

Description

Problem statement / motivation

Part of #291
Status: Ready for implementation (builds on typed provider contract established in #293 and #294)
Scope: Tavily provider integration, configuration plumbing, and tests

Tavily is a premier search engine purpose-built for AI agents and LLM applications (used by LangChain, AutoGen, and CrewAI):

  • Agent-optimized content: Cleans webpage content, filters out SEO spam/clickbait, and provides high-density, factual snippets directly optimized for LLM context windows.
  • Native domain filtering: Native server-side support for include_domains and exclude_domains.
  • News and raw content support: Direct support for news topic filtering and answer synthesis.
  • Differences from You.com / Brave: Tavily uses POST /search with a JSON payload (rather than GET with query parameters), authenticates via api_key in body or Authorization: Bearer header, and structures results with max_results (up to 20).
Proposed solution
A. Provider Implementation (tool/web_search/tavily.rs)
  • Endpoint: POST {base_url}/search with Content-Type: application/json.
  • Authentication: Authorization: Bearer {api_key} header (or api_key in payload). Key resolved from TAVILY_API_KEY.
  • Deserialization: Forward-compatible structs with #[serde(default)] mapping Tavily's response envelope (results: [{ title, url, content, published_date }]) to WebSearchProviderResponse.
  • Request body payload:
{
  "query": "...",
  "search_depth": "basic",
  "max_results": 10,
  "topic": "general",
  "include_domains": [],
  "exclude_domains": []
}
B. Input Adaptation Policy Matrix
Parameter / Input Tavily Behavior Gateway Adaptation Policy
allowed_domains / blocked_domains Native server-side support (include_domains / exclude_domains) Forwarded directly in JSON body; fallback client-side DomainFilter as a defense-in-depth guarantee.
count max_results (1..=20) Mapped to max_results; clamp to 20 with tracing::debug!.
freshness days parameter for recency (1, 7, 30) or time_range Map Freshness::Day -> days: 1, Week -> days: 7, Month -> days: 30, or best-effort date range mapping.
News topic: "news" When news is requested or present, set topic: "news".
You.com specifics (livecrawl, etc.) Tavily has its own include_raw_content option Ignored for base tool compatibility with tracing::debug!.
C. Concurrency & Rate Limiting
  • Concurrency Ceiling: Configurable via AGENTIC_WEB_SEARCH_MAX_CONCURRENT_QUERIES, defaulting to gateway limit or tier allowance.
  • 429 Handling: Fail web_search_call without retry, surfacing Retry-After verbatim per RFC 7231.
  • Auth Errors (401/403): Report failure explicitly naming TAVILY_API_KEY without leaking credentials or echoing response body.
D. Configuration & Precedence Matrix
Setting Environment Variable config.toml Key Default Value
Provider AGENTIC_WEB_SEARCH_PROVIDER [web_search].provider "you"
API Key Named by api_key_env [web_search].api_key_env "TAVILY_API_KEY" (for Tavily)
Base URL AGENTIC_WEB_SEARCH_BASE_URL [web_search].base_url https://api.tavily.com
Max Concurrency AGENTIC_WEB_SEARCH_MAX_CONCURRENT_QUERIES [web_search].max_concurrent_queries Inherits gateway limit
Scope Boundaries
In Scope Explicitly Out of Scope (Deferred)
TavilySearchProvider implementation (POST /search) include_raw_content / large body scraping payloads
✅ Native domain allow/block list forwarding ❌ Tavily Q&A answer injection (include_answer)
✅ News topic adaptation & count clamping ❌ Webhook / asynchronous search polling
✅ Axum mock HTTP tests (tests/web_search_tavily_test.rs) ❌ Custom reranking scoring thresholds
Verification Checklist
  • 200 OK (Search & News): Verify JSON POST payload shapes correctly and responses map to WebSearchResult.
  • Domain Filtering: Verify native include_domains/exclude_domains JSON fields are populated.
  • 401 / 403 Authentication: Fails web_search_call, references TAVILY_API_KEY, does not leak credentials.
  • 429 Rate Limit: Surfaces Retry-After verbatim without retrying.
  • Metadata: Echoes "provider": "tavily" in metadata[].
  • Zero Regressions: You.com and Brave Search remain byte-identical and unaffected.
  • Hygiene: cargo fmt, cargo clippy --all-targets -- -D warnings, and file size limits pass.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the typed provider contract established in issues #293 and #294, then inspect tool/web_search/tavily.rs and the existing provider implementations. Run or extend tests/web_search_tavily_test.rs using Axum mock HTTP tests. Done means the documented request mappings, authentication and error handling, metadata, domain filtering, and regression checks pass with cargo fmt and clippy.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.