ruvnet / ruvnet/agentic-flow

hooks metrics prints 'undefined' in text while --json returns 45 patterns / 6 memories; and 'Routing accuracy' counts post-edit calls, not routes — hooks route records nothing

Open
#201 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
812
Forks
175
Avg merge
2m
Merged PRs (30d)
3

Description

Summary

Two independent problems in hooks metrics:

  1. The text output prints undefined for Patterns / Memories / Error patterns, while --json returns the correct values from the same invocation. The data is present; the text renderer reads fields that do not exist.
  2. "Routing accuracy" does not measure routing. hooks route never records a route — the counter stays at 0 no matter how many routing decisions are made. The only writer is hooks post-edit, so the figure reported as routing accuracy is really "the fraction of edits the caller passed --success for".

Version: agentic-flow 2.1.2, Node v22.23.0, macOS 15.

1. undefined in text, correct values in JSON

Same command, same moment, same store:

$ npx agentic-flow hooks metrics

📚 Learning:
   Patterns: undefined
   Memories: undefined
   Error patterns: undefined

💚 Health: NEEDS-DATA
$ npx agentic-flow hooks metrics --json
learning.patternsLearned = 45
learning.memoriesStored  = 6
memory.total             = 6
memory.byType.project    = 6
errors.total             = 0
topPatterns              = [5 items]

Both agree with what is on disk:

.agentic-flow/intelligence.json → patterns 45, memories 6

So the JSON path reads the store correctly and the text path prints undefined — it appears to be looking for patterns / memories rather than patternsLearned / memoriesStored. topPatterns (5 entries of real data) has no text representation at all.

This matters because the text output is the default. A user running hooks metrics after a successful pretrain sees undefined and NEEDS-DATA and concludes the store is empty, when it holds 45 patterns.

2. hooks route does not record routes

metrics() { python3 -c "import json;print(json.load(open('.agentic-flow/intelligence.json'))['metrics'])"; }

metrics                                             # {'totalRoutes': 0, 'successfulRoutes': 0, 'routingHistory': []}
npx agentic-flow hooks route "fix a bug in the parser"
metrics                                             # {'totalRoutes': 0, ...}   ← unchanged

npx agentic-flow hooks post-edit src/roster.cjs --success --agent coder
metrics
# {'totalRoutes': 1, 'successfulRoutes': 1,
#  'routingHistory': [{'task': 'edit:src/roster.cjs', 'agent': 'coder', 'success': True}]}

hooks route produced a recommendation and recorded nothing. The single history entry comes from post-edit, and its shape shows what is actually being counted:

  • task: "edit:src/roster.cjs" — a file edit, not a routing request
  • agent: "coder" — supplied by the caller via --agent, not chosen by the router
  • success: true — supplied by the caller via --success

None of those three fields is a routing decision or an evaluation of one. So Routing accuracy: 100% after a successful edit means "the caller said the edit worked", and a user could drive that number to 100% or 0% by passing --success or --fail, without the router ever being consulted.

Consequently the Total routes: 0 shown above is accurate but misleading — routes were made, they are simply never counted.

Suggested fix

  1. Fix the text renderer to read patternsLearned / memoriesStored (and surface topPatterns, which is currently JSON-only).
  2. Record routing decisions in hooks route, so totalRoutes counts routes.
  3. Either separate the two counters — routing decisions vs edit outcomes — or rename the metric to reflect what it measures. As it stands, "Routing accuracy" is a caller-controlled edit success rate.

A test asserting that the text and --json outputs report the same numbers for the same store would catch item 1, and asserting totalRoutes increments after hooks route would catch item 2.

Related

  • #186 — hooks route: confidence fixed at 95.0%, factors NaN, alternatives outrank the recommendation. It also, per this issue, records nothing.
  • #198 — hooks explain: [object Object] in text output while --json carries the real structured data. Same class of text-renderer defect.
  • #185, #199, #200 — other hooks subcommands whose reported numbers disagree with what is on disk.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the hooks metrics, hooks route, and hooks post-edit entry points, comparing text and --json output against .agentic-flow/intelligence.json; src/roster.cjs is the example edit path. Add or update tests for matching text/JSON learning values and for route counts changing after hooks route. Done means text shows the stored metrics and routing metrics represent routing decisions rather than caller-reported edit outcomes.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.