google / google/adk-python

Native integration with AWS Bedrock LLMs

Đang mở
#6,249 1 bình luận 0 reaction 2 người được giao Được @xuanyang15 nhận Xem trên GitHub
models needs review
Ngôn ngữ chính
Python
Star
21.5k
Fork
4k
Merge trung bình
1 ngày 14 giờ
Pull request đã merge (30 ngày)
37

Mô tả

# Feature Request: Native integration with AWS Bedrock LLMs

## Required Information

### Is your feature request related to a specific problem?

ADK's native model support is currently limited to Google Gemini and a small number of third-party providers. This prevents AWS Bedrock users from seamlessly integrating Bedrock models, including Amazon Nova, Claude, Llama, and Mistral. While LiteLLM-based integration is available, it does not support bidirectional streaming for Speech-to-Speech (STS) models such as Nova Sonic. A native Bedrock integration would eliminate the limitations of intermediary layers, provide first-class support for Bedrock models and features, and ensure long-term compatibility as both ADK and AWS AI services evolve.

### Describe the Solution You'd Like

Add a first-class `BedrockLlm` provider that implements `BaseLlm` and `BaseLlmConnection`, supporting:

- **Text generation** (non-streaming) via Bedrock's `converse()` API
- **Text generation** (streaming) via Bedrock's `converse_stream()` API
- **Bidirectional streaming** (voice) via Bedrock's `invoke_model_with_bidirectional_stream()` API
- **Tool/function calling** with full schema support
- **Prompt caching** for system prompts and tool definitions
- **All Bedrock Converse-compatible models**: Amazon Nova, Anthropic Claude, Meta Llama, Mistral, Cohere, AI21 Labs, and cross-region variants

The provider would use the `aws-sdk-bedrock-runtime` Python SDK and standard AWS credential configuration (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION`).

**Usage example:**

```python
from google.adk.models.bedrock import BedrockLlm

# Text model
llm = BedrockLlm(model="amazon.nova-pro-v1:0")

# Voice model (bidirectional streaming)
llm = BedrockLlm(model="amazon.nova-2-sonic-v1:0")
```

### Impact on your work

This enables ADK adoption for developers and teams using AWS. Without native Bedrock support, users cannot leverage ADK with Bedrock models or must maintain custom integrations that break across ADK version upgrades.

### Willingness to contribute

Yes. We have a working implementation with full test coverage that is ready to be submitted as a PR.

---

## Recommended Information

### Describe Alternatives You've Considered

1. **Custom `BaseLlm` subclass per project** - Duplicates effort across users and is hard to keep in sync with ADK changes.
2. **LiteLLM** - Adds an extra hop, doesn't support bidirectional streaming for voice models, and introduces cold-start latency.

### Proposed API / Implementation

```python
from google.adk.models.bedrock import BedrockLlm, BedrockLlmConnection

# BedrockLlm implements BaseLlm
# - generate_content_async() -> uses converse() or converse_stream()
# - connect() -> returns BedrockLlmConnection for bidi streaming
# - supported_models() -> auto-detects Amazon Nova model IDs

# BedrockLlmConnection implements BaseLlmConnection
# - Manages HTTP/2 full-duplex stream lifecycle
# - Parses Bedrock events (audioOutput, textOutput, toolUse, etc.)
# - Handles interruptions and stream cancellation
```

Key implementation details:
- Singleton `BedrockRuntimeClient` shared across instances
- Tool schema normalization (type lowercasing, Decimal-to-float coercion)

### Additional Context

- Tested with Python 3.12 and 3.13
- Licensed under Apache-2.0 (same as ADK)
- All models available via Bedrock Converse API are supported by passing the model ID directly

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.