anthropics / anthropics/claude-code

[Feature Request] Add model performance monitoring and transparent API behavior documentation

Open
#95,300 0 comments 0 reactions 0 assignees View on GitHub
area:model enhancement
Dominant language
Python
Stars
145k
Forks
23.1k
PR merge metrics
PR metrics pending

Description

## Summary

When a newer model is released, older models (in this case Sonnet 5) appear to degrade sharply. I spent 4–5 hours stuck on a simple bug with Sonnet 5. Switching to Fable resolved it immediately.

If this is intentional routing or quality throttling, it is opaque and feels like a way to burn extra tokens. Please document expected model behavior and add a way to monitor quality regressions.

I have used Sonnet for more than six months across multiple accounts.

## What happened

1. I was debugging a simple issue in Claude Code using Sonnet 5.
2. The model looped, repeated itself, and failed to make progress for 4–5 hours.
3. I switched the model to Fable.
4. The same task completed successfully.

## Why this matters

- Large token spend with no useful output
- No signal that the model is underperforming
- No public documentation of how older models change after a new release
- Hard to trust Claude Code for production work if quality can drop without notice

## Requested outcome

- Transparent documentation of model versions, routing, and any known quality changes after a new release
- In-product monitoring or a warning when a session is looping / not making progress
- A way to pin a known-good model version so behavior does not silently change
- Confirmation that older models are not intentionally degraded

## Environment

- Platform: darwin
- Terminal: iTerm.app
- Version: 2.1.274
- Feedback ID: b0777965-efe8-403e-9a8b-e7b6e2c22a85

## Errors

```json
[]

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are named. Start by separating the requested documentation, quality-regression monitoring, looping warning, and model pinning into scoped requirements. Done should define supported model behavior and version semantics plus a testable monitoring or warning outcome, but the issue does not identify implementation locations.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, cli, documentation, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.