0xPlaygrounds / 0xPlaygrounds/rig

feat: Support Amazon Bedrock Mantle (OpenAI-compatible) for OpenAI-on-Bedrock models

Open
#2,189 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
8.6k
Forks
959
Avg merge
4h 32m
Merged PRs (30d)
117

Description

- [x] I have looked for existing issues (including closed) about this

## Feature Request
Add first-class support for **Amazon Bedrock Mantle** (OpenAI-compatible Chat Completions / Responses APIs) so Rig can call OpenAI-on-Bedrock models such as GPT OSS and other `openai.*` Mantle model ids.

Today `rig-bedrock` only speaks the Bedrock **Converse** API via `aws-sdk-bedrockruntime`. OpenAI models on Bedrock are primarily served through Mantle:

- Base URL shape: `https://bedrock-mantle.{region}.api.aws/openai/v1`
- Auth: Bedrock API key / short-term IAM bearer token (`Authorization: Bearer …`), not the Converse SDK client alone
- Model ids look like `openai.gpt-oss-120b`, `openai.gpt-oss-20b` (and other OpenAI-on-Mantle ids), not classic Converse FM ids alone

There is no existing issue covering Mantle. Closest related work is #1713 (`AWS_BEARER_TOKEN_BEDROCK`), which is about bearer auth for inference but does not introduce the Mantle OpenAI transport.

### Motivation
- OpenAI open-weight / OpenAI-on-Bedrock models are available on Mantle with OpenAI-compatible endpoints.
- Callers currently have to hand-roll an `openai::Client` with a custom base URL + token minting outside Rig.
- Converse model-constant additions alone do not cover Mantle Responses / Chat Completions usage (tools, reasoning params, etc.).

### Proposal
Extend `rig-bedrock` (preferred over a new companion crate) with a **Mantle path** that reuses Rig’s existing OpenAI-compatible client stack (Responses by default, matching `rig-core`’s OpenAI provider):

1. **Client construction**
- `Client` / builder helpers that produce an OpenAI-compatible client pointed at Mantle for a given AWS region.
- Auth options:
- Short-term IAM token generated from the default AWS credential chain (presign `CallWithBearerToken`, encode as `bedrock-api-key-…` — same algorithm as the official AWS Bedrock token generators).
- Optional long-lived / env bearer via `AWS_BEARER_TOKEN_BEDROCK` (can fully or partially address #1713 for the Mantle path).
2. **Model constants** for common Mantle OpenAI model ids (`openai.gpt-oss-20b`, `openai.gpt-oss-120b`, …). Free-form model strings remain supported.
3. **Docs + example** showing Mantle vs Converse: Converse stays the default for Claude / Nova / Llama; Mantle is the path for `openai.*` OpenAI-compatible models.
4. **Tests**: unit tests for token formatting / base URL; cassette-backed regression tests where feasible (same bar as other providers).

Non-goals for the initial PR:
- Replacing or rewriting the existing Converse client.
- Forcing model-id auto-routing magic that switches Converse vs Mantle based on string prefix (optional later; explicit Mantle client / constructor is clearer for v1).

### Alternatives
- Document “use `openai::Client` with a custom Mantle base URL” only (works today, but no first-class Bedrock auth, constants, or discoverability).
- Add Converse-only model id constants for OpenAI OSS runtime ids without Mantle (incomplete for Responses/Chat Completions and Mantle-only models).
- New companion crate `rig-bedrock-mantle` (extra surface area; Mantle is still Bedrock).

### References
- AWS Bedrock OpenAI models / Mantle endpoints: https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-openai.html
- Bedrock Mantle OpenAI-compatible base URL pattern: `https://bedrock-mantle.{region}.api.aws/openai/v1`
- Related: #1713

Happy to implement this against `rig-bedrock` following the OpenAI-compatible provider checklist in CONTRIBUTING.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.