azurenoops / azurenoops/spin_agent

[HIGH-1] Tool-ranker: always-1.0 sentinel + keyword-ladder routing survive

Open
#642 2 comments 0 reactions 0 assignees View on GitHub
area:routing bug priority:high
Dominant language
C#
Stars
3
Forks
1
Avg merge
18h 9m
Merged PRs (30d)
43

Description

## Problem

Two related routing defects remain open after PR #637 addressed token budgeting:

1. **Always-1.0 sentinel:** ~19–20 "core" tool prefixes are hardcoded to score `1.0`, effectively bypassing the ranker for a large fraction of tools. An artifact referenced as "72/130 tools" suggests this affects more than half the tool surface.
2. **Keyword-ladder routing:** `RouteToToolAsync` contains a 119-branch `if/else` ladder that operates as first-match with a silent default. This makes routing non-deterministic from a product perspective and impossible to test exhaustively.

## Evidence / Location

- Tool ranker: hardcoded `1.0` score block for core prefixes
- `RouteToToolAsync`: 119-branch keyword ladder, silent default path
- PR #637 closed budgeting but left routing logic untouched

## Fix

1. Remove or gate the always-1.0 sentinel. All tools should compete on real score except explicitly documented overrides with justification.
2. Refactor the 119-branch ladder into a data-driven dispatch table (dictionary/registry keyed by intent or tool ID) with an explicit fallback handler that logs unmatched routes.
3. Add routing unit tests that assert deterministic tool selection for representative prompts.

## Source Reviewers

Tony (M1) · Cyborg (F2) · Vision (#4) · Banner (#3) — all four independently flagged

---
> ⚠️ **Caveat:** All findings are static read-reviews; build/tests were not run. Reproduce before treating fix as confirmed.
> 📋 **Note:** War Machine's ATO/RMF findings are still pending and will be folded in as a second batch.

Contributor guide

Open the contributing guide

Research direction

Start by locating the hardcoded always-1.0 core-prefix block and RouteToToolAsync's 119-branch keyword ladder. Reproduce the behavior before changing it, then inspect existing routing tests or test conventions. Done means tools use documented scoring or overrides, dispatch has an explicit logged fallback, and representative prompts select deterministically.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
ai, backend-api-design
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.