api7 / api7/aisix

P1-7: Vertex AI / Google AI Studio native generateContent (currently OpenAI-shape relabel only)

Open
#59 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

gap-with-litellm P1
Dominant language
Rust
Stars
157
Forks
32
Avg merge
1h 25m
Merged PRs (30d)
145

Description

Problem

`crates/aisix-provider-gemini/src/lib.rs` is a one-line factory: `OpenAiBridge::new().with_name("gemini")` pointing at `generativelanguage.googleapis.com/v1beta/openai`. The native `:generateContent` API is explicitly out of scope (file header).

Native API exposes:

  • `tools` / `function_declarations` (different shape from OpenAI tools)
  • `system_instruction` (Gemini-specific)
  • `safety_settings` (categorical thresholds — separate from generic guardrails)
  • `generation_config` (top_k, response_mime_type, response_schema for structured output)

Customers using Gemini 2.x for structured output / function calling need the native shape.

Scope

DP (moonming/ai-gateway)
  • New crate `crates/aisix-provider-gemini-native/` (or extend existing crate) implementing Bridge against `v1beta/models/{model}:generateContent` + `:streamGenerateContent`.
  • Two sub-providers: Vertex AI (regional endpoint, OAuth2 service-account) + Google AI Studio (api-key in header).
  • Translate OpenAI chat-completion request → Gemini contents/parts shape (and back for response).
  • Map system messages → `system_instruction` field.
  • Map OpenAI `tools` → Gemini `tools.function_declarations` (recursive JSON schema translation).
  • Per-call `generation_config` from request `response_format` field.
CP (api7/api7ee-3-control-plane)
  • Provider enum: add `vertex_ai` and `google_ai_studio` (or just `gemini_native` with sub-mode).
  • Credential schema: Vertex needs service-account JSON; Studio needs API key.
Dashboard UI (api7/AISIX-Cloud)
  • Provider picker adds the native variants.
  • Credential form per variant.
e2e
  • Tool-calling test against mock Gemini.
  • structured-output test (`response_schema`).

Estimate

DP 5-7d, CP 1d, UI 1d, e2e 1d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with crates/aisix-provider-gemini/src/lib.rs and the Bridge contract, then trace how provider crates, control-plane enums, dashboard credentials, and e2e tests are organized. Done means native Vertex AI and Google AI Studio generateContent and streamGenerateContent support, request/response mappings, credentials and provider variants, plus mock tool-calling and structured-output tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, rust
Domain
ai, backend-api-design, cloud, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.