api7 / api7/aisix

[follow-up] audio / images / completions handlers dispatch via Provider enum, not Hub

Open
#368 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

cross-repo enhancement priority-normal
Dominant language
Rust
Stars
157
Forks
32
Avg merge
1h 25m
Merged PRs (30d)
145

Description

Follow-up to api7/AISIX-Cloud#417 close (api7/ai-gateway#365) — surfaced by audit MEDIUM-3.

Gap

The chat / chat_stream / embed / messages / responses path goes through Hub::dispatch_two_tier and benefits from the new Adapter::Openai family bridge fallthrough. But three other proxy handlers still go through the legacy require_provider(model) -> Provider flow + provider.default_base_url():

  • crates/aisix-proxy/src/audio.rs:295,418
  • crates/aisix-proxy/src/images.rs
  • crates/aisix-proxy/src/completions.rs

Even with Provider::Xai now in the enum (so the chat path works), a Model loaded with a brand-new long-tail vendor (not yet added to the enum but admitted by cp-api via provider_metadata) would:

  • chat → routes via Adapter::Openai family bridge → works ✅
  • audio/images/completions → require_provider returns an unknown variant → schema validation actually catches this first (model_schema's closed enum), so the symptom is "Model row never loads" rather than a 5xx at request time

But once #302 Phase A opens the model_schema provider field to free-form string (the cleanup item I listed in api7/AISIX-Cloud#302), audio/images/completions handlers will start returning enum-mismatch errors that the chat path no longer does.

Fix sketch

Refactor require_provider / resolve_base_url to also consult ProviderKey.api_base first and fall back to the legacy Provider::default_base_url() only when populated. Or stop dispatching off Model.provider for audio/images/completions and route those through the Hub family bridge too (parity with chat).

Why not in #365

  • The chat path is the customer-blocking surface from #417; audio/images is not exercised by the #417 repro
  • The fix is non-trivial (different handler structure than chat) and the right pattern depends on Phase A's broader refactor direction
  • Documented here so it doesn't get lost when #302 Phase A flips model_schema provider to free-form

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with require_provider and resolve_base_url, then read the handler paths in crates/aisix-proxy/src/audio.rs at lines 295 and 418, crates/aisix-proxy/src/images.rs, and crates/aisix-proxy/src/completions.rs. Compare their dispatch with the Hub::dispatch_two_tier chat path and review ProviderKey.api_base and Provider::default_base_url. Done means these handlers remain compatible with free-form providers after model_schema Phase A without losing legacy fallback behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.