aws-samples / aws-samples/sample-autonomous-cloud-coding-agents

feat(agent): default to Claude Opus 5 on the global Bedrock inference profile

Open
#741 3 comments 0 reactions 1 assignee Claimed by @scottschreckengaust View on GitHub
agent-runtime approved enhancement infra-cdk P1
Dominant language
TypeScript
Stars
143
Forks
46
Avg merge
3d 9h
Merged PRs (30d)
20

Description

## Summary

Move the platform default model from Claude Opus 4.8 to **Claude Opus 5**, and switch the cross-region inference profile from the hardcoded US geo to the **global** profile. Depends on / pairs with #740 (documentation of the model-configuration layers), which describes the same code paths.

Two changes that must land together: the model ID and the geo prefix. Opus 5's Bedrock profile ID observed in practice is `global.anthropic.claude-opus-5` — global-prefixed, not `us.`-prefixed — so bumping the model without also making the geo configurable would produce an IAM grant that does not cover the ID the agent actually invokes.

## Model IDs

| Form | ID |
|---|---|
| Bare foundation-model (Bedrock) | `anthropic.claude-opus-5` |
| Global cross-region inference profile | `global.anthropic.claude-opus-5` |
| US cross-region inference profile | `us.anthropic.claude-opus-5` |

> **Verify before merging.** These follow the Bedrock convention (`anthropic.` provider prefix; geo prefix for the cross-region profile) and `global.anthropic.claude-opus-5` is the form observed in use, but confirm against `aws bedrock list-inference-profiles` / `list-foundation-models` in the target account and Region — and confirm the account has **model access** for Opus 5 (IAM `grantInvoke` is not sufficient; Anthropic first-time-use and Marketplace prerequisites still apply). If Opus 5 is not yet entitled in the deployment account, this issue is blocked on access, not on code.

There is existing precedent for global profiles in this repo's tooling: `.threat-composer/20260526-1924/config/config.json` pins `global.anthropic.claude-opus-4-6-v1`.

## Why global instead of `us.`

`global.` routes to any supported commercial AWS Region, giving higher throughput and better resilience during peak demand than a US-only profile — which matters for a workload whose tasks run up to 8h and burst. The tradeoff is data residency: a deployer with a residency requirement needs a geo profile (`us.`/`eu.`/`apac.`) instead, which is exactly why this should be a **configurable** default rather than a second hardcoded constant.

`@aws-cdk/aws-bedrock-alpha` already models this: `CrossRegionInferenceProfileRegion.GLOBAL` (`= 'global'`) exists alongside `US`, `EU`, `APAC`, `JP`, `AU`, `US_GOV`, and `CrossRegionInferenceProfile` builds its `inferenceProfileId` as `` `${geoRegion}.${model.modelId}` `` — so `GLOBAL` needs no new abstraction, only threading.

## Code changes

### 1. Model default — four call sites, one commit

| File | Line | Change |
|---|---|---|
| `agent/src/config.py` | 563 | `ANTHROPIC_MODEL` fallback → `global.anthropic.claude-opus-5` |
| `agent/src/models.py` | 157 | `anthropic_model` field default → same |
| `cdk/src/constructs/bedrock-models.ts` | 34 | add `anthropic.claude-opus-5` to `DEFAULT_BEDROCK_MODEL_IDS` (**bare**, no geo prefix) |
| `cli/src/repo-display.ts` | 48 | `PLATFORM_REPO_DEFAULTS.model_id` → same as the agent default (this also drives what `platform doctor` probes for access) |

`cdk/test/constructs/bedrock-models.test.ts:83` is a drift guard that regex-greps `agent/src/config.py` for the `ANTHROPIC_MODEL` fallback and asserts the bare form is in `DEFAULT_BEDROCK_MODEL_IDS`. **It currently asserts `agentDefault` matches `/^us\./` and strips only a `us.` prefix — it will fail on a `global.`-prefixed default.** Generalize the prefix to `/^(us|eu|apac|global)\./` as part of this change; do not weaken the assertion to a bare `.*` (the point is to catch a bare ID being used where a profile ID is required).

Decide whether to keep `anthropic.claude-opus-4-8` in the grant list. Recommend **keeping** it: blueprints may pin it per-repo, and dropping it would break those repos at turn 0 with `AccessDenied`. Removing 4.8 should be a separate, announced change.

### 2. Geo region — make it configurable, not a second hardcode

Both grant sites hardcode US today:

- `cdk/src/stacks/agent.ts:549` — `geoRegion: bedrock.CrossRegionInferenceProfileRegion.US`
- `cdk/src/constructs/ecs-agent-cluster.ts:585` — string-concatenates `` `us.${modelId}` `` into the inference-profile ARN

Add a **`bedrockGeoRegion` CDK context key** (default `us` to preserve existing behavior for current deployers; set to `global` in `cdk/cdk.json` as part of this change) resolved in `cdk/src/constructs/bedrock-models.ts` alongside `resolveBedrockModelIds`, and thread it into both sites. Map the string to `CrossRegionInferenceProfileRegion` in `agent.ts`; use it in place of the literal `us.` in `ecs-agent-cluster.ts`. Reject unknown values at synth, consistent with how `resolveBedrockModelIds` already fails loudly on a malformed override.

Context (not a CloudFormation parameter) because this value must be available at **synth** — it participates in `grantInvoke` / ARN construction. A CFN parameter resolves after synth and would force `Resource: '*'`, undoing the deliberate per-model IAM scoping documented at `bedrock-models.ts:29-32`. See #740 for the fuller strategy rationale.

### 3. Fix the prefix guard (latent bug, independent of Opus 5)

`resolveBedrockModelIds` rejects `us|eu|apac`-prefixed entries (`bedrock-models.ts:84`) but **not** `global.`. So `-c bedrockModels='[\"global.anthropic.claude-opus-5\"]'` today silently yields an invalid `us.global.anthropic.claude-opus-5` ARN instead of failing at synth. Add `global` (and ideally `us-gov`, `jp`, `au`) to the guard regex. Note the guard's error message and the new `bedrockGeoRegion` key interact: `bedrockModels` stays bare-IDs-only; the geo comes from the new key.

### 4. Auxiliary model

`ANTHROPIC_DEFAULT_HAIKU_MODEL` is hardcoded to `us.anthropic.claude-haiku-4-5-20251001-v1:0` at `cdk/src/stacks/agent.ts:393`. If the deployment moves to `global`, this stays `us.`-prefixed and becomes inconsistent — the two profiles would route differently. Either derive its prefix from `bedrockGeoRegion` too, or explicitly document why the auxiliary model stays pinned to US. Prefer deriving it.

### 5. Local runner

`agent/run.sh:208` passes `-e ANTHROPIC_MODEL=${ANTHROPIC_MODEL:-us.anthropic.claude-sonnet-4-6}`, which **overrides** the `config.py` default rather than deferring to it — so local Docker runs would still be on Sonnet 4.6 after this change. #740 already proposes making `run.sh` pass the variable through only when the caller set it; that fix is a prerequisite for local and deployed runs agreeing on Opus 5.

## Prompt / behavioral considerations

A model bump is not purely mechanical — the agent's prompts and effort settings are tuned for the current model. Before flipping the default, run a representative task set (at minimum: `coding/new-task-v1` and `coding/pr-review-v1`) and check for:

- **Turn/token budget shifts** — `max_turns` defaults (100) and any budget gating may need re-baselining; re-measure rather than applying a multiplier.
- **Prompt over-prescription** — scaffolding tuned for an older model can *reduce* output quality on a newer one. A/B the pipeline prompts in `agent/src/` with step-by-step scaffolding removed.
- **Code-review recall** — if `coding/pr-review-v1` instructs the model to be conservative or to report only high-severity findings, newer models follow that more literally and measured recall can drop even as bug-finding improves. Prefer report-everything-with-confidence + filter downstream.
- **Longer turns** — plan for the possibility that individual requests take materially longer; check timeouts in the agent runtime and the orchestrator's poll/await paths before assuming a hang is a bug.

## Acceptance criteria

- [ ] Opus 5 model IDs verified against `aws bedrock list-inference-profiles` / `list-foundation-models` in the target Region, and account model access confirmed
- [ ] All four default call sites updated in one commit (`config.py`, `models.py`, `bedrock-models.ts`, `repo-display.ts`)
- [ ] `bedrock-models.test.ts:83` drift guard generalized to accept a `global.` prefix and still passing
- [ ] `bedrockGeoRegion` context key added, defaulted, threaded into **both** grant sites, and validated at synth; `cdk.json` sets it to `global`
- [ ] `resolveBedrockModelIds` prefix guard rejects `global.` (and `us-gov`/`jp`/`au`)
- [ ] `ANTHROPIC_DEFAULT_HAIKU_MODEL` prefix derived from the geo key, or its US pinning explicitly justified in a comment
- [ ] `agent/run.sh` no longer injects a hardcoded model fallback (#740)
- [ ] Deployed smoke test: one task per compute backend (`agentcore` and `ecs`) completes end-to-end and opens a PR — this is what proves the IAM grant actually covers the invoked profile ARN
- [ ] `mise run build` green; `cdk/test/stacks/agent.test.ts` and `cdk/test/constructs/ecs-agent-cluster.test.ts` ARN assertions updated
- [ ] Docs updated per #740 (do not land a new default while the docs still say Sonnet 4.6)

## Related

- #740 — documentation of the model-configuration layers and env vars (should land first, or concurrently)
- #489 — tagged Bedrock application inference profiles for cost attribution; overlaps with the profile-ARN plumbing touched here

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.