api7 / api7/aisix

P0-2: AWS Bedrock provider (chat + embeddings + image)

Open
#44 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

cross-repo gap-with-litellm P0
Dominant language
Rust
Stars
157
Forks
32
Avg merge
1h 25m
Merged PRs (30d)
145

Description

Problem

LiteLLM ships native AWS Bedrock support (chat / streaming / embeddings / image-gen / Guardrails). ai-gateway has zero — no aisix-provider-bedrock crate.

Bedrock is one of the largest enterprise LLM surfaces; missing it is a hard blocker for every AWS-native customer pitch.

Scope

DP (moonming/ai-gateway)
  • New crate crates/aisix-provider-bedrock/ implementing the Bridge trait.
  • AWS SigV4 signing (use aws-sigv4 + aws-credential-types).
  • Endpoints: POST /model/{modelId}/invoke (sync) + /invoke-with-response-stream (SSE).
  • Translate OpenAI-shape chat-completions ↔ Bedrock Converse API.
  • Per-model adapter for the Bedrock model families that have non-Converse formats (Anthropic-on-Bedrock / Llama / Titan / Cohere / Stability / Mistral) — start with Anthropic-on-Bedrock since the request shape is closest.
  • Embeddings: Titan Embed v1/v2 + Cohere Embed.
  • Image: Stability SD3 + Titan Image.
CP (api7/api7ee-3-control-plane)
  • Provider enum extended in ai_model.provider (currently openai/anthropic/gemini/deepseek/mock); add bedrock.
  • Credential schema must support AWS access key + secret + region + session token (vs OpenAI's single api_key). Tied to #2625 credential resource being org-scoped.
Dashboard UI (api7/AISIX-Cloud)
  • Provider picker adds Bedrock.
  • Credential form: AWS access key / secret / region (+ optional session token / role ARN for cross-account).
  • Model form: Bedrock modelId picker (preload common ones from model_prices_and_context_window.json-equivalent).
e2e
  • Mock Bedrock with localstack or a SigV4-aware mock; assert the SigV4 signature + the request body format on the wire.

Estimate

DP 5-7d, CP 2d, UI 1.5d, e2e 1.5d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the proposed crates/aisix-provider-bedrock/ and its Bridge trait, then review the Bedrock invoke and streaming endpoints and the AWS SigV4 requirements. Track the related control-plane provider enum and credential schema, Dashboard provider and credential forms, and the e2e localstack or SigV4-aware mock. Done means chat, streaming, embeddings, image generation, provider configuration, UI support, and wire-level signature and body assertions are covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, rust
Domain
api, backend, cloud, frontend, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.