hooks metrics prints 'undefined' in text while --json returns 45 patterns / 6 memories; and 'Routing accuracy' counts post-edit calls, not routes — hooks route records nothing
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 812
- Forks
- 175
- Avg merge
- 2m
- Merged PRs (30d)
- 3
Description
Summary
Two independent problems in hooks metrics:
- The text output prints
undefinedfor Patterns / Memories / Error patterns, while--jsonreturns the correct values from the same invocation. The data is present; the text renderer reads fields that do not exist. - "Routing accuracy" does not measure routing.
hooks routenever records a route — the counter stays at 0 no matter how many routing decisions are made. The only writer ishooks post-edit, so the figure reported as routing accuracy is really "the fraction of edits the caller passed--successfor".
Version: agentic-flow 2.1.2, Node v22.23.0, macOS 15.
1. undefined in text, correct values in JSON
Same command, same moment, same store:
$ npx agentic-flow hooks metrics
📚 Learning:
Patterns: undefined
Memories: undefined
Error patterns: undefined
💚 Health: NEEDS-DATA
$ npx agentic-flow hooks metrics --json
learning.patternsLearned = 45
learning.memoriesStored = 6
memory.total = 6
memory.byType.project = 6
errors.total = 0
topPatterns = [5 items]
Both agree with what is on disk:
.agentic-flow/intelligence.json → patterns 45, memories 6
So the JSON path reads the store correctly and the text path prints undefined — it appears to be looking for patterns / memories rather than patternsLearned / memoriesStored. topPatterns (5 entries of real data) has no text representation at all.
This matters because the text output is the default. A user running hooks metrics after a successful pretrain sees undefined and NEEDS-DATA and concludes the store is empty, when it holds 45 patterns.
2. hooks route does not record routes
metrics() { python3 -c "import json;print(json.load(open('.agentic-flow/intelligence.json'))['metrics'])"; }
metrics # {'totalRoutes': 0, 'successfulRoutes': 0, 'routingHistory': []}
npx agentic-flow hooks route "fix a bug in the parser"
metrics # {'totalRoutes': 0, ...} ← unchanged
npx agentic-flow hooks post-edit src/roster.cjs --success --agent coder
metrics
# {'totalRoutes': 1, 'successfulRoutes': 1,
# 'routingHistory': [{'task': 'edit:src/roster.cjs', 'agent': 'coder', 'success': True}]}
hooks route produced a recommendation and recorded nothing. The single history entry comes from post-edit, and its shape shows what is actually being counted:
task: "edit:src/roster.cjs"— a file edit, not a routing requestagent: "coder"— supplied by the caller via--agent, not chosen by the routersuccess: true— supplied by the caller via--success
None of those three fields is a routing decision or an evaluation of one. So Routing accuracy: 100% after a successful edit means "the caller said the edit worked", and a user could drive that number to 100% or 0% by passing --success or --fail, without the router ever being consulted.
Consequently the Total routes: 0 shown above is accurate but misleading — routes were made, they are simply never counted.
Suggested fix
- Fix the text renderer to read
patternsLearned/memoriesStored(and surfacetopPatterns, which is currently JSON-only). - Record routing decisions in
hooks route, sototalRoutescounts routes. - Either separate the two counters — routing decisions vs edit outcomes — or rename the metric to reflect what it measures. As it stands, "Routing accuracy" is a caller-controlled edit success rate.
A test asserting that the text and --json outputs report the same numbers for the same store would catch item 1, and asserting totalRoutes increments after hooks route would catch item 2.
Related
- #186 —
hooks route: confidence fixed at 95.0%, factorsNaN, alternatives outrank the recommendation. It also, per this issue, records nothing. - #198 —
hooks explain:[object Object]in text output while--jsoncarries the real structured data. Same class of text-renderer defect. - #185, #199, #200 — other
hookssubcommands whose reported numbers disagree with what is on disk.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the hooks metrics, hooks route, and hooks post-edit entry points, comparing text and --json output against .agentic-flow/intelligence.json; src/roster.cjs is the example edit path. Add or update tests for matching text/JSON learning values and for route counts changing after hooks route. Done means text shows the stored metrics and routing metrics represent routing decisions rather than caller-reported edit outcomes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, typescript
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100