aws-samples / aws-samples/sample-multi-agent-orchestration-chat-on-agentcore
Add Google Gemma 4 model support (Bedrock Mantle endpoint)
- Dominant language
- TypeScript
- Stars
- 128
- Forks
- 12
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 4
Description
## Summary
Google DeepMind's Gemma 4 family is now available on Amazon Bedrock (announced June 10/15, 2026):
| Model | Bedrock Model ID | Architecture | Context Window | Key Use Case |
|-------|-----------------|--------------|----------------|--------------|
| Gemma 4 31B | `google.gemma-4-31b` | Dense, 30.7B params | 256K tokens | Reasoning & coding |
| Gemma 4 26B-A4B | `google.gemma-4-26b-a4b` | MoE, 25.2B total / 3.8B active | 256K tokens | Cost- & latency-sensitive |
| Gemma 4 E2B | `google.gemma-4-e2b` | Dense, 5.1B / 2.3B effective (PLE) | 128K tokens | Low-latency interactive |
All three variants support:
- ✅ Native function calling (client-side tool calling)
- ✅ Response streaming
- ✅ Reasoning (built-in)
- ✅ Structured output
- ✅ Multimodal input (text, image)
- ✅ 35+ languages
**Available regions**: us-east-1 (N. Virginia), us-east-2 (Ohio), us-west-2 (Oregon), eu-central-1 (Frankfurt).
> **References**: [AWS What's New](https://aws.amazon.com/about-aws/whats-new/2026/06/gemma-4-amazon-bedrock/) · [AWS Weekly Roundup — June 15, 2026](https://aws.amazon.com/blogs/aws/aws-weekly-roundup-aws-finops-agent-in-preview-gemma-4-on-bedrock-kiro-pro-max-and-more-june-15-2026) · [Bedrock docs — Google models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-cards-google.html)
---
## ⚠️ Architectural Note: Bedrock Mantle Endpoint (Not Converse API)
Unlike all existing Moca models (Claude, Nova, Qwen3), **Gemma 4 is only available on the `bedrock-mantle` endpoint**, which exposes an OpenAI-compatible API at:
```
https://bedrock-mantle.{region}.api.aws/openai/v1
```
This is architecturally distinct from the Bedrock Converse API used by `BedrockModel` in `@strands-agents/sdk`. Gemma 4 models:
- Have **no cross-region inference profile prefix** (no `global.`/`us.`/`eu.` — In-Region only)
- Show **Geo inference ID: Not supported** and **Global inference ID: Not supported** in the Bedrock console
- Use a **bearer-token** authentication scheme (not Bedrock SigV4 directly)
> This is the same infrastructure pattern as the OpenAI GPT models on Bedrock (see #19), but routed through `bedrock-mantle` rather than the Bedrock Responses API.
---
## ✅ Good News: Strands SDK Already Has Native Mantle Support
The current dependency `@strands-agents/sdk ^1.2.0` already ships an `OpenAIModel` provider with first-class `bedrockMantleConfig` support:
```ts
import { OpenAIModel } from '@strands-agents/sdk/models/openai';
const model = new OpenAIModel({
api: 'chat',
modelId: 'google.gemma-4-31b',
bedrockMantleConfig: { region: 'us-east-1' },
});
```
The SDK automatically:
1. Derives the Mantle `baseURL` from the region
2. Lazily loads `@aws/bedrock-token-generator` (optional peer dep) and mints short-lived bearer tokens from the ambient AWS credentials — tokens are refreshed automatically so long-running agents survive the token lifetime
3. Handles streaming via the OpenAI Chat Completions wire format
**No custom model class needs to be written.** The implementation cost is materially lower than the OpenAI GPT path in #19.
---
## Required Changes
### 1. `packages/libs/core/src/bedrock-models.ts` — Model registry (Single Source of Truth)
- Add `'Google'` to the `provider` union type in `BedrockModelDefinition`:
```ts
readonly provider: 'Anthropic' | 'Amazon' | 'Qwen' | 'Google';
```
- Add a new `usesMantleApi?: boolean` flag to `BedrockModelDefinition` to allow downstream routing without string-matching model IDs:
```ts
readonly usesMantleApi?: boolean;
```
- Export a new helper `usesMantleApi(modelId: string): boolean` mirroring the `getModelRegion` / `isReasoningCapable` pattern.
- Add the three Gemma 4 entries (all In-Region, region-pinned to `us-east-1` as primary):
```ts
{
id: 'google.gemma-4-31b',
name: 'Gemma 4 31B',
provider: 'Google',
maxOutputTokens: 8192, // ⚠ confirm from Bedrock quotas page
region: 'us-east-1',
usesMantleApi: true,
},
{
id: 'google.gemma-4-26b-a4b',
name: 'Gemma 4 26B-A4B',
provider: 'Google',
maxOutputTokens: 8192, // ⚠ confirm
region: 'us-east-1',
usesMantleApi: true,
},
{
id: 'google.gemma-4-e2b',
name: 'Gemma 4 E2B',
provider: 'Google',
maxOutputTokens: 8192, // ⚠ confirm
region: 'us-east-1',
usesMantleApi: true,
},
```
- Add tests to `src/__tests__/bedrock-models.test.ts`.
### 2. `packages/agent/src/config/bedrock.ts` — Model factory
Extend `createBedrockModel()` to route Mantle model IDs to `OpenAIModel`:
```ts
import { OpenAIModel } from '@strands-agents/sdk/models/openai';
import { usesMantleApi, getModelRegion } from '@moca/core';
export function createBedrockModel(options?: BedrockModelOptions): BedrockModel | OpenAIModel {
const modelId = options?.modelId || config.BEDROCK_MODEL_ID;
if (usesMantleApi(modelId)) {
const region = options?.region || getModelRegion(modelId) || config.BEDROCK_REGION;
return new OpenAIModel({
api: 'chat',
modelId,
maxTokens: options?.maxTokens ?? getMaxOutputTokens(modelId),
bedrockMantleConfig: { region },
});
}
// existing Converse-API path unchanged ...
}
```
Notes:
- `reasoningEffort` / `cacheConfig` should be omitted for Mantle models (not supported on this endpoint).
- Reasoning via the Chat Completions API does NOT return reasoning tokens to the client (OpenAI spec limitation). The depth selector should be hidden for Gemma 4 (since `reasoningCapable` is not set).
### 3. `packages/agent/package.json` — New dependency
Add `@aws/bedrock-token-generator` as a direct dependency (currently it is an optional peer dep of the Strands SDK and must be explicitly installed by the consumer):
```json
"@aws/bedrock-token-generator": "^1.1.0"
```
### 4. `packages/frontend/src/config/models.ts` — Frontend provider type
Add `'Google'` to the union and `VALID_PROVIDERS` set:
```ts
export interface BedrockModel {
id: string;
name: string;
provider: 'Anthropic' | 'Amazon' | 'Qwen' | 'Google';
}
```
### 5. `packages/cdk/config/environment-types.ts` — CDK provider type
Add `'Google'` to the `BedrockModelConfig.provider` union.
### 6. `packages/cdk/config/environment-utils.ts` — CDK infrastructure config
- Add `'Google'` to `VALID_PROVIDERS`.
- Add the three Gemma 4 entries to `DEFAULT_CONFIG.bedrockModels`:
```ts
{ id: 'google.gemma-4-31b', name: 'Gemma 4 31B', provider: 'Google', region: 'us-east-1' },
{ id: 'google.gemma-4-26b-a4b', name: 'Gemma 4 26B-A4B', provider: 'Google', region: 'us-east-1' },
{ id: 'google.gemma-4-e2b', name: 'Gemma 4 E2B', provider: 'Google', region: 'us-east-1' },
```
- Review `deriveBedrockIamResources()`: The `google.*` IDs have no inference-profile prefix (same pattern as `qwen.*`), so only foundation-model ARNs are generated. However, the **Mantle endpoint uses bearer-token auth** — verify which IAM action(s) `@aws/bedrock-token-generator` requires to mint tokens and ensure the agent Lambda execution role grants them. See open question #2 below.
---
## Known Limitations (from Bedrock docs)
| Limitation | Impact on Moca |
|-----------|---------------|
| **Parallel tool calls not supported** — only one tool call per turn | Moca's agent loop will work but multi-tool responses in a single turn will fail; test and add a guardrail if needed |
| **Reasoning tokens not returned by Chat Completions API** | Reasoning panel in UI will be empty; depth selector is not shown (`reasoningCapable` not set) |
| **Max request payload 3.5 MB** (including images/video) for Gemma 4 31B | Document in README; large multimodal inputs may need resizing |
| **In-Region only** — no cross-region inference profile | Region must be one of: `us-east-1`, `us-east-2`, `us-west-2`, `eu-central-1` |
| **Prompt caching not supported** | `ENABLE_PROMPT_CACHING` flag already safely no-ops for non-Anthropic models via the SDK's `auto` strategy |
---
## Open Questions
1. **Exact `maxOutputTokens`**: The Bedrock model card pages for Gemma 4 do not yet list a specific max output token limit. Verify via the [Bedrock quotas page](https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html) or a live inference test before committing values to the registry.
2. **IAM permissions for Mantle bearer-token generation**: `@aws/bedrock-token-generator` mints bearer tokens from AWS credentials. Which IAM action(s) does it require? Must be confirmed and added to CDK's `deriveBedrockIamResources()` or the agent Lambda's IAM policy directly.
3. **`OpenAIModel` return type in `createBedrockModel`**: The function currently returns `BedrockModel`. Changing the signature to `BedrockModel | OpenAIModel` may require callers to update type annotations. Verify that `OpenAIModel` implements the same `Model` interface and whether the union propagates cleanly.
4. **Parallel tool call rejection**: If Moca's agent loop sends a parallel tool-call request to a Gemma 4 model, verify whether the Strands SDK or the Mantle API returns a graceful error and add a guardrail if needed.
5. **Session persistence / content-block codec**: Moca's session serializer (`content-block-codec.ts`) stores messages in the Bedrock Converse native shape. OpenAI Chat Completions messages have a different structure. Verify whether session persistence round-trips correctly for Gemma 4 conversations, or whether a separate serialization path is needed.
6. **Region pin override**: Default pin is `us-east-1`. Teams deploying Moca in `us-east-2`, `us-west-2`, or `eu-central-1` may want a closer-region pin. Consider documenting how to override per `environments.ts`.
---
## Files to Modify
| File | Change Type |
|------|-------------|
| `packages/libs/core/src/bedrock-models.ts` | Modify — add `'Google'` provider, `usesMantleApi` flag + helper, 3 model entries |
| `packages/libs/core/src/__tests__/bedrock-models.test.ts` | Modify — add tests for new flag/helper and Gemma 4 entries |
| `packages/agent/src/config/bedrock.ts` | Modify — routing to `OpenAIModel` for Mantle models |
| `packages/agent/src/config/__tests__/bedrock.test.ts` | Modify — add routing tests |
| `packages/agent/package.json` | Modify — add `@aws/bedrock-token-generator` dependency |
| `packages/frontend/src/config/models.ts` | Modify — add `'Google'` to provider union + `VALID_PROVIDERS` |
| `packages/cdk/config/environment-types.ts` | Modify — add `'Google'` to `BedrockModelConfig.provider` |
| `packages/cdk/config/environment-utils.ts` | Modify — add `'Google'` provider, Gemma 4 entries, verify IAM |
Contributor guide
Research direction
Start with packages/libs/core/src/bedrock-models.ts and its tests, then inspect packages/agent/src/config/bedrock.ts and its tests for the existing model factory. Review the frontend and CDK configuration files listed in the issue, including IAM resource derivation and session serialization. Done means Gemma 4 models are registered, routed through Mantle, permitted in configuration, and covered by passing tests with the open questions resolved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- backend, cloud, frontend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100