aws-samples / aws-samples/sample-multi-agent-orchestration-chat-on-agentcore

Add Google Gemma 4 model support (Bedrock Mantle endpoint)

Open
#47 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
TypeScript
Stars
128
Forks
12
Avg merge
3d 1h
Merged PRs (30d)
4

Description

## Summary

Google DeepMind's Gemma 4 family is now available on Amazon Bedrock (announced June 10/15, 2026):

| Model | Bedrock Model ID | Architecture | Context Window | Key Use Case |
|-------|-----------------|--------------|----------------|--------------|
| Gemma 4 31B | `google.gemma-4-31b` | Dense, 30.7B params | 256K tokens | Reasoning & coding |
| Gemma 4 26B-A4B | `google.gemma-4-26b-a4b` | MoE, 25.2B total / 3.8B active | 256K tokens | Cost- & latency-sensitive |
| Gemma 4 E2B | `google.gemma-4-e2b` | Dense, 5.1B / 2.3B effective (PLE) | 128K tokens | Low-latency interactive |

All three variants support:
- ✅ Native function calling (client-side tool calling)
- ✅ Response streaming
- ✅ Reasoning (built-in)
- ✅ Structured output
- ✅ Multimodal input (text, image)
- ✅ 35+ languages

**Available regions**: us-east-1 (N. Virginia), us-east-2 (Ohio), us-west-2 (Oregon), eu-central-1 (Frankfurt).

> **References**: [AWS What's New](https://aws.amazon.com/about-aws/whats-new/2026/06/gemma-4-amazon-bedrock/) · [AWS Weekly Roundup — June 15, 2026](https://aws.amazon.com/blogs/aws/aws-weekly-roundup-aws-finops-agent-in-preview-gemma-4-on-bedrock-kiro-pro-max-and-more-june-15-2026) · [Bedrock docs — Google models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-cards-google.html)

---

## ⚠️ Architectural Note: Bedrock Mantle Endpoint (Not Converse API)

Unlike all existing Moca models (Claude, Nova, Qwen3), **Gemma 4 is only available on the `bedrock-mantle` endpoint**, which exposes an OpenAI-compatible API at:

```
https://bedrock-mantle.{region}.api.aws/openai/v1
```

This is architecturally distinct from the Bedrock Converse API used by `BedrockModel` in `@strands-agents/sdk`. Gemma 4 models:
- Have **no cross-region inference profile prefix** (no `global.`/`us.`/`eu.` — In-Region only)
- Show **Geo inference ID: Not supported** and **Global inference ID: Not supported** in the Bedrock console
- Use a **bearer-token** authentication scheme (not Bedrock SigV4 directly)

> This is the same infrastructure pattern as the OpenAI GPT models on Bedrock (see #19), but routed through `bedrock-mantle` rather than the Bedrock Responses API.

---

## ✅ Good News: Strands SDK Already Has Native Mantle Support

The current dependency `@strands-agents/sdk ^1.2.0` already ships an `OpenAIModel` provider with first-class `bedrockMantleConfig` support:

```ts
import { OpenAIModel } from '@strands-agents/sdk/models/openai';

const model = new OpenAIModel({
api: 'chat',
modelId: 'google.gemma-4-31b',
bedrockMantleConfig: { region: 'us-east-1' },
});
```

The SDK automatically:
1. Derives the Mantle `baseURL` from the region
2. Lazily loads `@aws/bedrock-token-generator` (optional peer dep) and mints short-lived bearer tokens from the ambient AWS credentials — tokens are refreshed automatically so long-running agents survive the token lifetime
3. Handles streaming via the OpenAI Chat Completions wire format

**No custom model class needs to be written.** The implementation cost is materially lower than the OpenAI GPT path in #19.

---

## Required Changes

### 1. `packages/libs/core/src/bedrock-models.ts` — Model registry (Single Source of Truth)

- Add `'Google'` to the `provider` union type in `BedrockModelDefinition`:
```ts
readonly provider: 'Anthropic' | 'Amazon' | 'Qwen' | 'Google';
```
- Add a new `usesMantleApi?: boolean` flag to `BedrockModelDefinition` to allow downstream routing without string-matching model IDs:
```ts
readonly usesMantleApi?: boolean;
```
- Export a new helper `usesMantleApi(modelId: string): boolean` mirroring the `getModelRegion` / `isReasoningCapable` pattern.
- Add the three Gemma 4 entries (all In-Region, region-pinned to `us-east-1` as primary):
```ts
{
id: 'google.gemma-4-31b',
name: 'Gemma 4 31B',
provider: 'Google',
maxOutputTokens: 8192, // ⚠ confirm from Bedrock quotas page
region: 'us-east-1',
usesMantleApi: true,
},
{
id: 'google.gemma-4-26b-a4b',
name: 'Gemma 4 26B-A4B',
provider: 'Google',
maxOutputTokens: 8192, // ⚠ confirm
region: 'us-east-1',
usesMantleApi: true,
},
{
id: 'google.gemma-4-e2b',
name: 'Gemma 4 E2B',
provider: 'Google',
maxOutputTokens: 8192, // ⚠ confirm
region: 'us-east-1',
usesMantleApi: true,
},
```
- Add tests to `src/__tests__/bedrock-models.test.ts`.

### 2. `packages/agent/src/config/bedrock.ts` — Model factory

Extend `createBedrockModel()` to route Mantle model IDs to `OpenAIModel`:

```ts
import { OpenAIModel } from '@strands-agents/sdk/models/openai';
import { usesMantleApi, getModelRegion } from '@moca/core';

export function createBedrockModel(options?: BedrockModelOptions): BedrockModel | OpenAIModel {
const modelId = options?.modelId || config.BEDROCK_MODEL_ID;

if (usesMantleApi(modelId)) {
const region = options?.region || getModelRegion(modelId) || config.BEDROCK_REGION;
return new OpenAIModel({
api: 'chat',
modelId,
maxTokens: options?.maxTokens ?? getMaxOutputTokens(modelId),
bedrockMantleConfig: { region },
});
}

// existing Converse-API path unchanged ...
}
```

Notes:
- `reasoningEffort` / `cacheConfig` should be omitted for Mantle models (not supported on this endpoint).
- Reasoning via the Chat Completions API does NOT return reasoning tokens to the client (OpenAI spec limitation). The depth selector should be hidden for Gemma 4 (since `reasoningCapable` is not set).

### 3. `packages/agent/package.json` — New dependency

Add `@aws/bedrock-token-generator` as a direct dependency (currently it is an optional peer dep of the Strands SDK and must be explicitly installed by the consumer):

```json
"@aws/bedrock-token-generator": "^1.1.0"
```

### 4. `packages/frontend/src/config/models.ts` — Frontend provider type

Add `'Google'` to the union and `VALID_PROVIDERS` set:

```ts
export interface BedrockModel {
id: string;
name: string;
provider: 'Anthropic' | 'Amazon' | 'Qwen' | 'Google';
}
```

### 5. `packages/cdk/config/environment-types.ts` — CDK provider type

Add `'Google'` to the `BedrockModelConfig.provider` union.

### 6. `packages/cdk/config/environment-utils.ts` — CDK infrastructure config

- Add `'Google'` to `VALID_PROVIDERS`.
- Add the three Gemma 4 entries to `DEFAULT_CONFIG.bedrockModels`:
```ts
{ id: 'google.gemma-4-31b', name: 'Gemma 4 31B', provider: 'Google', region: 'us-east-1' },
{ id: 'google.gemma-4-26b-a4b', name: 'Gemma 4 26B-A4B', provider: 'Google', region: 'us-east-1' },
{ id: 'google.gemma-4-e2b', name: 'Gemma 4 E2B', provider: 'Google', region: 'us-east-1' },
```
- Review `deriveBedrockIamResources()`: The `google.*` IDs have no inference-profile prefix (same pattern as `qwen.*`), so only foundation-model ARNs are generated. However, the **Mantle endpoint uses bearer-token auth** — verify which IAM action(s) `@aws/bedrock-token-generator` requires to mint tokens and ensure the agent Lambda execution role grants them. See open question #2 below.

---

## Known Limitations (from Bedrock docs)

| Limitation | Impact on Moca |
|-----------|---------------|
| **Parallel tool calls not supported** — only one tool call per turn | Moca's agent loop will work but multi-tool responses in a single turn will fail; test and add a guardrail if needed |
| **Reasoning tokens not returned by Chat Completions API** | Reasoning panel in UI will be empty; depth selector is not shown (`reasoningCapable` not set) |
| **Max request payload 3.5 MB** (including images/video) for Gemma 4 31B | Document in README; large multimodal inputs may need resizing |
| **In-Region only** — no cross-region inference profile | Region must be one of: `us-east-1`, `us-east-2`, `us-west-2`, `eu-central-1` |
| **Prompt caching not supported** | `ENABLE_PROMPT_CACHING` flag already safely no-ops for non-Anthropic models via the SDK's `auto` strategy |

---

## Open Questions

1. **Exact `maxOutputTokens`**: The Bedrock model card pages for Gemma 4 do not yet list a specific max output token limit. Verify via the [Bedrock quotas page](https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html) or a live inference test before committing values to the registry.

2. **IAM permissions for Mantle bearer-token generation**: `@aws/bedrock-token-generator` mints bearer tokens from AWS credentials. Which IAM action(s) does it require? Must be confirmed and added to CDK's `deriveBedrockIamResources()` or the agent Lambda's IAM policy directly.

3. **`OpenAIModel` return type in `createBedrockModel`**: The function currently returns `BedrockModel`. Changing the signature to `BedrockModel | OpenAIModel` may require callers to update type annotations. Verify that `OpenAIModel` implements the same `Model` interface and whether the union propagates cleanly.

4. **Parallel tool call rejection**: If Moca's agent loop sends a parallel tool-call request to a Gemma 4 model, verify whether the Strands SDK or the Mantle API returns a graceful error and add a guardrail if needed.

5. **Session persistence / content-block codec**: Moca's session serializer (`content-block-codec.ts`) stores messages in the Bedrock Converse native shape. OpenAI Chat Completions messages have a different structure. Verify whether session persistence round-trips correctly for Gemma 4 conversations, or whether a separate serialization path is needed.

6. **Region pin override**: Default pin is `us-east-1`. Teams deploying Moca in `us-east-2`, `us-west-2`, or `eu-central-1` may want a closer-region pin. Consider documenting how to override per `environments.ts`.

---

## Files to Modify

| File | Change Type |
|------|-------------|
| `packages/libs/core/src/bedrock-models.ts` | Modify — add `'Google'` provider, `usesMantleApi` flag + helper, 3 model entries |
| `packages/libs/core/src/__tests__/bedrock-models.test.ts` | Modify — add tests for new flag/helper and Gemma 4 entries |
| `packages/agent/src/config/bedrock.ts` | Modify — routing to `OpenAIModel` for Mantle models |
| `packages/agent/src/config/__tests__/bedrock.test.ts` | Modify — add routing tests |
| `packages/agent/package.json` | Modify — add `@aws/bedrock-token-generator` dependency |
| `packages/frontend/src/config/models.ts` | Modify — add `'Google'` to provider union + `VALID_PROVIDERS` |
| `packages/cdk/config/environment-types.ts` | Modify — add `'Google'` to `BedrockModelConfig.provider` |
| `packages/cdk/config/environment-utils.ts` | Modify — add `'Google'` provider, Gemma 4 entries, verify IAM |

Contributor guide

Open the contributing guide

Research direction

Start with packages/libs/core/src/bedrock-models.ts and its tests, then inspect packages/agent/src/config/bedrock.ts and its tests for the existing model factory. Review the frontend and CDK configuration files listed in the issue, including IAM resource derivation and session serialization. Done means Gemma 4 models are registered, routed through Mantle, permitted in configuration, and covered by passing tests with the open questions resolved.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
backend, cloud, frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.