hoangsonww / hoangsonww/Claude-Code-Agent-Monitor
[Feature]: Statistical session health scoring with per-project anomaly baselines
- Dominant language
- TypeScript
- Stars
- 1k
- Forks
- 234
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 18
Description
### What problem does this solve?
A session that has gone wrong looks exactly like a session that is working hard. Both show a green `working` badge, a rising token count, and a steady event stream. The difference — an agent stuck in an edit/test/edit loop, re-reading the same file 40 times, retrying a failing command, or burning cache writes after a bad compaction — is only visible if you sit and watch the Activity Feed, which defeats the point of a dashboard.
The existing alerting engine can catch the crude cases (no events for N minutes, `working` for over an hour, tokens past a fixed number), but every one of those thresholds is a global constant the user has to guess. "Working for 45 minutes" is alarming for a one-file docs fix and completely normal for a large refactor. Fixed thresholds on a workload this variable produce either noise or silence.
### Proposed solution
Derive the baseline from the user's own history instead of asking them to guess it, and score live sessions against it. Deliberately **statistical and local — no model calls, no network**.
**Baselines** — a periodic job computes rolling per-project distributions (p50/p90/p99) over the last N completed sessions for a handful of cheap metrics already present in `events` and `token_usage`:
- events per minute, and tool calls per minute
- distinct-files-touched vs. total file-tool calls (a *thrash ratio*)
- error-shaped events as a fraction of tool calls
- consecutive repeats of the same `(tool_name, target)` pair
- cache-write tokens as a fraction of total input
- wall-clock duration and total cost
Baselines are stored in a small `session_baselines` table keyed by project + metric, recomputed on a timer, and fall back to a global baseline until a project has enough completed sessions to be meaningful.
**Scoring** — a live session gets per-metric z-scores against its project baseline, combined into a 0–100 health score with a plain-English reason list: *"Repeating `Read src/db.js` — 12 consecutive calls (p99 for this project is 3)"*, *"Error rate 34% of tool calls (p90: 6%)"*. The reasons are the feature; the number is just how you sort by it.
**Surfacing** — a health chip on `SessionCard` and `SessionDetail` (green/amber/red plus the top reason), a "Needs attention" sort on the Sessions page, and a new `anomaly` alert rule type so a degrading session can page you through the webhook/push path that already exists. Cold-start behavior is explicit: no chip at all until the project has a baseline, rather than a fabricated score.
**Honesty about false positives** — every score is explainable and dismissible per session, and dismissals feed back as a signal that the heuristic is mis-tuned for that project.
### Alternatives considered
- **Ask an LLM to judge the session** (#8). Different tool for a different job: #8 writes a narrative *after* the fact and costs tokens per call. This runs continuously, costs nothing, works offline, and is meant to interrupt you *during* a run. They compose well — the anomaly reasons would be excellent input to an AI narrative.
- **More fixed-threshold alert rules.** That's the status quo; the guessing problem is the complaint.
- **A single opaque health score.** Rejected — an unexplained number that occasionally turns red gets ignored within a week.
### Area
Analytics / Tokens
### How important is this to you?
Would significantly improve my workflow
### Additional context
Every metric above is computable from the current schema with no new ingestion. The main implementation risk is cost of recomputation on large histories — the baseline job should be incremental and capped, in the spirit of the transcript-cache and token-reconciliation work in #294 / #296.
Contributor guide
Assessment
This issue has not been assessed yet.