api7 / api7/aisix

feat: Voyage AI rerank support on /v1/rerank with top_n/top_k + results/data adapter (#213 Phase 2.5)

Open
#228 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement gap-with-litellm P2 priority-normal
Dominant language
Rust
Stars
157
Forks
32
Avg merge
1h 25m
Merged PRs (30d)
145

Description

Summary

PR #227 (#213 Phase 1) added Cohere on /v1/rerank. The Phase 2 PR adds Jina (also identity-mapped to the Cohere wire shape).

Voyage AI also exposes /v1/rerank at https://api.voyageai.com with Bearer auth, but its wire shape differs from Cohere/OpenAI/Jina in two material places:

  1. Request — top_k instead of top_n: per https://docs.voyageai.com/reference/reranker-api, Voyage's reranker accepts top_k (integer) for the result-count cap. Cohere/Jina/OpenAI-compat use top_n. The gateway today forwards the body verbatim, so a caller sending {top_n: 5} would have Voyage return 400 unknown field "top_n".

  2. Response — data instead of results: Voyage returns {object, data: [{index, relevance_score}], usage, model} whereas Cohere/Jina return {results: [{index, relevance_score, document?}], ...}. Caller code that iterates response.results would break on a Voyage upstream.

Both differences require a small request/response adapter that's out of scope for the rest of #213's surgical "relax the gate + add default api_base" pattern.

Suggested fix shape

In crates/aisix-proxy/src/rerank.rs::dispatch, after the existing model field rewrite and before the upstream POST:

// Per #213 Phase 2.5: Voyage's /v1/rerank diverges from the
// Cohere/Jina/OpenAI-compat wire shape on two fields. Translate
// caller-side names to Voyage's names on the request, then
// translate Voyage's response field names back on the way out.
if model.provider == Some(Provider::Voyage) {
    if let Some(map) = body.as_object_mut() {
        if let Some(top_n) = map.remove("top_n") {
            map.insert("top_k".to_string(), top_n);
        }
    }
}

And on the response side, after body_bytes:

if model.provider == Some(Provider::Voyage) {
    let mut json: serde_json::Value = serde_json::from_slice(&body_bytes)?;
    if let Some(obj) = json.as_object_mut() {
        if let Some(data) = obj.remove("data") {
            obj.insert("results".to_string(), data);
        }
    }
    body_bytes = serde_json::to_vec(&json)?.into();
}

(Sketch — actual implementation needs to handle the byte-stream → JSON parse → re-serialize round-trip carefully, and the response transform should preserve usage / model / other fields verbatim.)

Tests

Mirror the Cohere/Jina pattern:

  • Rust unit test voyage_provider_dispatches_with_field_translation — pin caller's top_n becomes upstream top_k; pin upstream's data becomes caller-visible results.
  • E2E case in rerank-e2e.test.tsVoyage provider: gateway translates field names on both request and response (#213 Phase 2.5).

Updates required alongside

  • Add Provider::Voyage enum variant + default_base_url/as_str arms.
  • Add "voyage" to JSON schema enum.
  • Add Voyage to default_base_for_provider in rerank.rs (https://api.voyageai.com).
  • Update gate to accept {Openai, Cohere, Jina, Voyage}.
  • Update docs §4.7 to list Voyage as supported, document the wire-shape adapter.

References

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in crates/aisix-proxy/src/rerank.rs::dispatch and compare the existing Cohere/Jina dispatch tests. Add Voyage provider support, request and response field translation, the default URL, schema and gate updates, and documentation §4.7; run voyage_provider_dispatches_with_field_translation and the named Voyage case in rerank-e2e.test.ts to verify completion.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.