ruvnet / ruvnet/ruflo

Static TASK_PATTERNS preempt a stronger keyword-overlap match, costing ~4pp routing accuracy

Open
#2,886 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
72.7k
Forks
8.6k
Avg merge
2d 23h
Merged PRs (30d)
83

Description

`hooks route` picks the first semantic match clearing `score > 0.4`. Static
patterns need only that bar; learned patterns additionally need
`support >= 2 && reliability >= 0.75` (hooks-tools.js, the #2864 comment).

The problem is not the learned gate — it is that a static pattern winning on
score **skips `suggestAgentsForTask` entirely**, and that function contains a
second, much stronger learned stage: nearest-neighbour over
`routing-outcomes.json` by keyword overlap (>= 2 required). A static pattern
matching on a weak char-hash similarity therefore beats an outcome match with
14 overlapping keywords.

**Measured** on a 75-outcome memorization replay (ruflo 3.33.0, both backends):

| decided by | accuracy | n |
|---|---|---|
| keyword-fallback | 53% | 57 |
| learned-* | 67% | 9 |
| **static TASK_PATTERNS** | **33%** | **9** |

Static is the least accurate path and wins 9 routes. Simulating
`suggestAgentsForTask` on those exact 9 tasks (same KEYWORD_PATTERNS table,
same stopword tokenizer, same >=2 overlap rule):

```
static today : 3/9 correct
if fallback : 6/9 correct net +3 routes
overall : 39/75 (52%) -> 42/75 (56%)
```

The three flips, all currently lost to a static pattern:

```
want tester static gave security-architect fallback: tester (KEYWORD:test)
want tester static gave architect fallback: tester (OUTCOME-OVERLAP:7)
want coder static gave architect fallback: coder (OUTCOME-OVERLAP:14)
```

An outcome match with 14 shared keywords losing to `refactor-task` is the
clearest case.

**Suggested fix:** require a static pattern to beat the best
`suggestAgentsForTask` result rather than short-circuiting it — or gate static
patterns on a higher score bar than learned ones, since they carry no
support/reliability evidence. Either restores the ordering the #2864 fix was
reaching for.

**Secondary:** `buildLearnedRoutingPatterns` accepts `LearnedRoutingOptions`,
but the only caller (`loadLearnedPatterns`) passes none and no env/config
override exists, so `minKeywordSupport`, `minDiscriminativeShare` and the `0.4`
bar cannot be tuned without a code change.

Environment: ruflo v3.33.0, macOS, 75 labelled outcomes. Identical results on
native HNSW and `CLAUDE_FLOW_DISABLE_NATIVE_ROUTER=1`.

Contributor guide

Open the contributing guide

Research direction

Start in hooks-tools.js by tracing the static TASK_PATTERNS path and suggestAgentsForTask, then inspect routing-outcomes.json and the loadLearnedPatterns/buildLearnedRoutingPatterns call chain. Reproduce the 75-outcome comparison described in the issue and verify that static matches no longer bypass the stronger learned result, with routing accuracy improving from 52% toward 56%.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.