aws-samples / aws-samples/sample-agent-cost-bench
Model IDs error for Claude and Codex
- Dominant language
- Python
- Stars
- 74
- Forks
- 19
- Avg merge
- 30m
- Merged PRs (30d)
- 2
Description
**Error**
```
agent_cost_bench cli-compare run 20260713_034129_8d0fca
Comparison: Opus 4.8 (Kiro / Claude Code / Codex)
Tasks: 14 × CLIs: 3 = 42 runs
CLIs: Kiro (claude-opus-4.8), Claude Code (claude-opus-4.8), OpenAI Codex (gpt-5.5)
Concurrency: 3 parallel run(s) (strategy: per_target)
· running bedrock-sentiment / Kiro (claude-opus-4.8) (vibe) …
· running bedrock-sentiment / Claude Code (claude-opus-4.8) (vibe) …
· running bedrock-sentiment / OpenAI Codex (gpt-5.5) (vibe) …
[ 1/42] ✗ failed bedrock-sentiment / Claude Code (claude-opus-4.8) $0.0000 2s (5s elapsed)
```
Current Model IDs:
> Claude Code — model_id: us.anthropic.claude-opus-4-8
> OpenAI Codex — model_id: gpt-5.5
Correct model IDs
> Claude Code — model_id: global.anthropic.claude-opus-4-8
> OpenAI Codex — model_id: openai.gpt-5.5
[config.cli-compare.yaml](https://github.com/user-attachments/files/29951617/config.cli-compare.yaml)
Contributor guide
Research direction
Start with config.cli-compare.yaml and the agent_cost_bench cli-compare run command shown in the report. Check the Claude Code and OpenAI Codex model_id entries, update them to the stated values, and rerun the comparison to confirm both models no longer fail from incorrect IDs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100