aws-samples / aws-samples/sample-autonomous-cloud-coding-agents

Docs: document model-ID configuration layers, governing env vars, and global Bedrock endpoints

Open
#740 4 comments 0 reactions 1 assignee Claimed by @scottschreckengaust View on GitHub
agent-runtime approved documentation infra-cdk P1
Dominant language
TypeScript
Stars
143
Forks
46
Avg merge
3d 9h
Merged PRs (30d)
20

Description

## Doc area

Developer guide (`docs/guides/DEVELOPER_GUIDE.md`), User guide (`docs/guides/USER_GUIDE.md`), Design (`docs/design/REPO_ONBOARDING.md`), `agent/README.md`.

## Describe the issue

There is no single place that answers "how do I change the model the agent uses?" The model ID is set in **five** places across three languages, each governing a different layer, and the docs describe neither the layering nor the precedence. Three of the documented defaults are also stale, and the region/geo prefix is hardcoded so **global Bedrock inference endpoints cannot be selected at all**.

### The layers as they actually exist today

| Layer | Mechanism | Where | Scope |
|---|---|---|---|
| 1. IAM invoke allowlist | CDK context `bedrockModels` (array of **bare** foundation-model IDs) → `resolveBedrockModelIds` | `cdk/src/constructs/bedrock-models.ts:34,48` | Which models the role *may* invoke. Does **not** select a model. |
| 2. Platform default model | Python literal only — no CDK/env knob | `agent/src/config.py:563`, `agent/src/models.py:157` | Fallback when a repo pins none. **Editing source is the only way to change it.** |
| 3. Auxiliary model | Stack-level env `ANTHROPIC_DEFAULT_HAIKU_MODEL`, hardcoded | `cdk/src/stacks/agent.ts:393`; default `agent/src/config.py:569` | WebFetch summarization / pre-flight. |
| 4. Per-repo override | Blueprint `agent.modelId` → RepoTable `model_id` → ECS injects `ANTHROPIC_MODEL`; AgentCore passes payload `model_id` | `cdk/src/constructs/blueprint.ts:85`, `handlers/shared/repo-config.ts:37`, `handlers/shared/strategies/ecs-strategy.ts:217` | Per repository. |
| 5. Per-task / local | Payload `model_id` (aliased to `anthropic_model`, `agent/src/pipeline.py:1780`); locally `ANTHROPIC_MODEL` shell env | `agent/run.sh:208` | Single run. |

Plus a workflow-pinned model validated against `WORKFLOW_MODEL_ALLOWLIST` (`cdk/src/handlers/shared/workflows.ts:84`), and a CLI-side mirror of the platform default at `cli/src/repo-display.ts:48` that also drives what `platform doctor` probes for access.

### Env-var reference that should exist but doesn't

| Variable | Set by | Governs | Prefix form |
|---|---|---|---|
| `ANTHROPIC_MODEL` | ECS strategy from blueprint `model_id`; shell for local `run.sh`; **never** set at stack level for AgentCore | Main model | Inference-profile ID (`us.anthropic.…`) |
| `ANTHROPIC_DEFAULT_HAIKU_MODEL` | `cdk/src/stacks/agent.ts:393` | Auxiliary/fast model | Inference-profile ID — **must** be `us.`-prefixed; a bare ID cannot be invoked on-demand for Claude 4.x |
| `CLAUDE_CODE_USE_BEDROCK=1` | `agent/run.sh` + runtime env | Routes the SDK to Bedrock | n/a |

The prefix asymmetry is the single biggest trap and is undocumented: **layers 2–5 use the `us.`-prefixed inference-profile ID; layer 1 (`bedrockModels`) takes the bare foundation-model ID** and derives the profile ARN itself. Passing `us.anthropic.…` to the context key would yield `us.us.anthropic.…`, so `resolveBedrockModelIds` rejects `us|eu|apac` prefixes at synth (`bedrock-models.ts:84`).

### Global Bedrock endpoints are unreachable

`@aws-cdk/aws-bedrock-alpha` exposes `CrossRegionInferenceProfileRegion.GLOBAL` (`= 'global'`), which routes to any supported commercial Region. ABCA hardcodes US in **both** grant sites:

- `cdk/src/stacks/agent.ts:549` — `geoRegion: bedrock.CrossRegionInferenceProfileRegion.US`
- `cdk/src/constructs/ecs-agent-cluster.ts:585` — string-concatenates `` `us.${modelId}` ``

So a `global.anthropic.…` model ID passed via blueprint `model_id` reaches the agent and is invoked, but the IAM grant never covers `inference-profile/global.anthropic.…` → `AccessDenied`. Related bug: the `resolveBedrockModelIds` prefix guard rejects `us|eu|apac` but **not** `global`, so `-c bedrockModels='[\"global.anthropic.…\"]'` silently produces an invalid `us.global.anthropic.…` ARN instead of failing at synth.

### Stale documented defaults

Code default is `us.anthropic.claude-opus-4-8` (`agent/src/config.py:563`). These still say Sonnet 4.6:

- `agent/README.md:122` and `:149`
- `docs/guides/DEVELOPER_GUIDE.md:250`
- `docs/design/REPO_ONBOARDING.md:123` — "Claude Sonnet 4", and attributes the source to "CDK stack props", which is wrong (it is a Python literal)
- `agent/run.sh:33,208` — a *behavioural* bug, not just docs: because `run.sh` always passes `-e ANTHROPIC_MODEL=${ANTHROPIC_MODEL:-us.anthropic.claude-sonnet-4-6}`, local Docker runs **override** the `config.py` default rather than deferring to it. Local and deployed runs silently use different models.

`agent/README.md:126` also documents `ANTHROPIC_DEFAULT_HAIKU_MODEL` with a **bare** default (`anthropic.claude-haiku-…`), contradicting the `us.`-prefix requirement asserted in `agent/src/models.py:162` and the value actually deployed at `agent/src/stacks/agent.ts:393`.

Only one cross-layer invariant is currently guarded: `cdk/test/constructs/bedrock-models.test.ts:83` regex-greps `agent/src/config.py` and asserts the agent's fallback is in `DEFAULT_BEDROCK_MODEL_IDS`. Nothing guards the CLI mirror, the haiku model, or the docs.

## Affected docs

- `docs/guides/DEVELOPER_GUIDE.md` (env-var table ~L248, Blueprint example ~L79, troubleshooting ~L265)
- `docs/guides/USER_GUIDE.md` (blueprint settings table ~L218)
- `docs/design/REPO_ONBOARDING.md` (defaults table ~L118)
- `agent/README.md` (env-var table ~L118)
- `agent/run.sh` (usage text L33 + the default at L208)

## Suggested change

### Recommended strategy: CDK context for deploy-time, blueprint for per-repo — *not* deploy-time CFN parameters or per-environment files

Three options were considered:

1. **CDK context (`cdk.json` / `-c`)** — resolves at **synth**, so it can drive both the IAM grant list *and* a stack-level env var from one value, and a typo fails synth loudly. Already the established pattern here (`bedrockModels`).
2. **CloudFormation parameters** — resolve at **deploy**, i.e. after synth. A CFN parameter therefore *cannot* participate in `resolveBedrockModelIds` or the `grantInvoke` ARN construction without falling back to `Resource: '*'`, which would undo the deliberate per-model IAM scoping documented in `bedrock-models.ts:29-32`. **Reject.**
3. **Per-environment config files** — real value only once dev/stage/prod diverge; today it adds a config surface without a consumer. **Defer.**

**Recommendation: extend the existing CDK-context pattern with two new keys, and document context as the one deploy-time knob.**

- **`defaultModelId`** (new, bare foundation-model ID) — becomes the single source for the platform default. Synth would (a) set `ANTHROPIC_MODEL` as a stack-level env var on **both** the AgentCore runtime and the ECS task definition, and (b) auto-union it into the grant list. This moves layer 2 out of a Python literal into a deploy-time knob, makes `agent/src/config.py` a pure last-resort fallback, and lets the regex-grep drift guard be replaced by a real assertion.
- **`bedrockGeoRegion`** (new, one of `us` | `eu` | `apac` | `global`, default `us`) — threaded into both grant sites to replace the hardcoded `CrossRegionInferenceProfileRegion.US` and the `` `us.${modelId}` `` concatenation. This is what makes global endpoints reachable. Non-US deployers currently have no supported path at all.
- **`bedrockModels`** keeps its current meaning: *widen the allowlist* beyond the default.
- Fix the prefix guard to also reject `global.` (and ideally `us-gov.`, `jp.`, `au.`).

**Precedence, to be documented explicitly** (narrowest wins):

```
per-task payload model_id
> blueprint agent.modelId (RepoTable model_id)
> stack env ANTHROPIC_MODEL (from CDK context defaultModelId)
> agent/src/config.py fallback
```
…all gated by the IAM allowlist from `bedrockModels` ∪ `defaultModelId`, which in turn is gated by account-level Bedrock model access (the outer gate — IAM `grantInvoke` alone is not sufficient; Anthropic first-time-use and Marketplace prerequisites still apply).

### Docs deliverables

1. A **single canonical section** — suggest `docs/guides/DEVELOPER_GUIDE.md`, linked from `USER_GUIDE.md` and `agent/README.md` — containing the layer table, the env-var table, the precedence chain, and the bare-vs-`us.`-prefix rule with a worked example of the `us.us.` failure mode.
2. A **global-endpoint subsection**: when to prefer `global.` (throughput/resilience across all commercial Regions) vs a geo profile (data-residency), the `bedrockGeoRegion` context key, and the caveat that geo choice must match the deployment Region's entitlements.
3. Correct the four stale Sonnet-4.6 references and the bare-vs-prefixed haiku default.
4. Fix `agent/run.sh` to **not** inject a hardcoded fallback — pass `ANTHROPIC_MODEL` through only when the caller set it, so local runs inherit the same default as deployed runs.
5. Extend the drift guard: assert `cli/src/repo-display.ts` `PLATFORM_REPO_DEFAULTS.model_id`, the agent default, and the grant list agree; and that the documented default matches code.

Items 1–3 are docs-only and can land first. Items 4–5 are small code changes; the `defaultModelId` / `bedrockGeoRegion` context keys are a separate implementation issue and should be split out if this issue is scoped to documentation only.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.