hooks route: Confidence is a fixed 95.0% on every task, all four routing factors are NaN, and the listed alternatives outrank the recommendation

Open
#186 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
typescript
Domain
ai, cli

Research direction

Run the provided npx agentic-flow hooks route reproduction, then inspect dist/mcp/fastmcp/tools/hooks/intelligence-bridge.js and dist/intelligence/RuVectorIntelligence.js at the reported initialization paths. Trace the MoEAttention and AdamOptimizer construction failures and how fallback results are formatted. Done means unavailable factors and degraded mode are explicit, and the returned recommendation is consistent with the candidate scores.

Written by the indexing model from the issue text.

Description

Summary

agentic-flow hooks route returns a constant recommendation — researcher at 95.0% — for every task, and all four of its reported routing factors are NaN%. The alternatives it lists score higher than the agent it recommends.

The cause is visible in the command's own stderr: the RuVector subsystems that produce those four factors fail to construct, the code falls back, and the fallback still prints a 95% confidence as though the intelligence layer had answered.

Version: agentic-flow 2.1.2, macOS 15 (darwin 25.6.0).

Reproduce

for t in "implement user authentication" "fix a failing unit test" "write documentation for the API"; do
  npx agentic-flow hooks route "$t" 2>/dev/null | grep -E 'Recommended|Confidence'
done

Output — three unrelated tasks, identical routing:

🎯 Recommended Agent: researcher   📊 Confidence: 95.0%
🎯 Recommended Agent: researcher   📊 Confidence: 95.0%
🎯 Recommended Agent: researcher   📊 Confidence: 95.0%

Full output for a single run:

🎯 Recommended Agent: researcher
📊 Confidence: 95.0%

📋 Routing Factors:
   • ruvector_intelligence: NaN%
   • sona-embedding: NaN%
   • hnsw-search: NaN%
   • moe-attention: NaN%

🔄 Alternatives:
   - optimizer (97%)
   - coordinator (97%)
   - test-engineer (96%)

⏱️  Latency: 65ms

Note the recommendation (95%) ranks below all three alternatives (97/97/96).

Root cause (from the command's own stderr)

hooks route prints these before the result — the run is not silent, but it exits 0 and presents the output as a successful routing:

[RuVectorIntelligence] Attention init failed, using fallback: TypeError: MoEAttention is not a constructor
    at RuVectorIntelligence.initializeAsync (dist/intelligence/RuVectorIntelligence.js:208:41)
    at new RuVectorIntelligence (dist/intelligence/RuVectorIntelligence.js:147:33)
    at createIntelligenceLayer (dist/intelligence/RuVectorIntelligence.js:845:12)
    at initializeIntelligence (dist/mcp/fastmcp/tools/hooks/intelligence-bridge.js:341:32)
    at routeTaskIntelligent (dist/mcp/fastmcp/tools/hooks/intelligence-bridge.js:384:32)

[RuVectorIntelligence] Optimizer init failed: TypeError: AdamOptimizer is not a constructor
    at RuVectorIntelligence.initializeAsync (dist/intelligence/RuVectorIntelligence.js:226:30)
    ... same chain ...

…and then, on the next line:

[IntelligenceBridge] RuVector intelligence layer initialized
[IntelligenceBridge] Features: Micro-LoRA, MoE Attention

So the bridge announces MoE Attention as an active feature immediately after MoEAttention is not a constructor. The four NaN% factors correspond exactly to the subsystems that failed to construct — they are unset values formatted as percentages.

Impact

  • Routing carries no signal. Every task gets researcher, so any consumer that acts on the recommendation is routing at random-constant, not intelligently.
  • The failure is presented as success: exit code 0, a confident 95.0%, and a feature banner claiming the failed subsystem is active. Only the NaN% values and the stack traces reveal otherwise, and a caller parsing the recommendation never sees them.
  • Alternatives outranking the recommendation is independently wrong — whatever the scores mean, the top-scoring entry is not the one returned.

Suggested fix

  1. Fail loudly, or clearly degrade: if the intelligence layer cannot construct, say so in the result (mode: fallback) rather than printing a 95% confidence and a feature list that includes the broken subsystem.
  2. Don't print NaN% — omit unavailable factors, or show them as unavailable.
  3. Return the highest-scoring candidate, or explain why the recommendation ranks below its own alternatives.
  4. The two TypeError: X is not a constructor failures look like import-shape mismatches (default vs named export) against the ruvector build — worth checking whether these ever construct successfully on any platform.

Related

  • #182 — write commands that report success and persist nothing.
  • #185 — hooks pretrain appends duplicate memories while under-reporting the store size.

Same family: the command exits 0 and reports a confident result that its own internals contradict.

Dominant language
TypeScript
Stars
812
Forks
175
Avg merge
2m
Merged PRs (30d)
3

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ruvnet/agentic-flow

All issues in ruvnet/agentic-flow

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.