aws-samples / aws-samples/sample-agent-cost-bench

Model IDs error for Claude and Codex

Open Beginner friendly
#1 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
74
Forks
19
Avg merge
30m
Merged PRs (30d)
2

Description

**Error**

```
agent_cost_bench cli-compare run 20260713_034129_8d0fca
Comparison: Opus 4.8 (Kiro / Claude Code / Codex)
Tasks: 14 × CLIs: 3 = 42 runs
CLIs: Kiro (claude-opus-4.8), Claude Code (claude-opus-4.8), OpenAI Codex (gpt-5.5)
Concurrency: 3 parallel run(s) (strategy: per_target)
· running bedrock-sentiment / Kiro (claude-opus-4.8) (vibe) …
· running bedrock-sentiment / Claude Code (claude-opus-4.8) (vibe) …
· running bedrock-sentiment / OpenAI Codex (gpt-5.5) (vibe) …
[ 1/42] ✗ failed bedrock-sentiment / Claude Code (claude-opus-4.8) $0.0000 2s (5s elapsed)
```

Current Model IDs:

> Claude Code — model_id: us.anthropic.claude-opus-4-8
> OpenAI Codex — model_id: gpt-5.5

Correct model IDs
> Claude Code — model_id: global.anthropic.claude-opus-4-8
> OpenAI Codex — model_id: openai.gpt-5.5

[config.cli-compare.yaml](https://github.com/user-attachments/files/29951617/config.cli-compare.yaml)

Contributor guide

Open the contributing guide

Research direction

Start with config.cli-compare.yaml and the agent_cost_bench cli-compare run command shown in the report. Check the Claude Code and OpenAI Codex model_id entries, update them to the stated values, and rerun the comparison to confirm both models no longer fail from incorrect IDs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.