NVIDIA-NeMo / NVIDIA-NeMo/Switchyard
[feature] Add optional de-escalation to the escalation router
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3.2k
- Forks
- 291
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 182
Description
Problem
The escalation router permanently latches a session to the strong tier after escalation. In multi-turn agent workloads, the difficult part may later be resolved, but every remaining turn still uses the strong model. This raises cost and prevents the router from returning routine work to the efficient tier.
Proposed solution
Add an optional de-escalation policy under the existing escalation configuration:
- Hold the strong tier for a configurable minimum number of turns.
- Judge completed strong-tier responses with an explicit strong-phase marker.
- Require consecutive safe-to-release verdicts before returning to the efficient tier.
- Optionally cap a strong-tier run and apply an efficient-tier cooldown to prevent immediate bouncing.
- Retain state through the existing session identity.
- Preserve the current permanent latch and packaged prompt exactly when the option is omitted.
The policy should fail safely: judge failures keep the strong tier, and a strong-target fallback must not be judged as a strong response or preserve a partial release streak.
Alternatives considered
- Keep permanent latching: simple, but it continues paying strong-tier cost after recovery.
- Route every turn independently: can bounce between tiers and lose useful provider cache locality.
- Use the stage router: useful for deterministic tool signals, but it does not replace response-based trajectory judging.
Scope notes
This belongs in the libsy escalation algorithm, with matching deployment-TOML, Rust, PyO3/Python, tests, and routing documentation. It requires no protocol changes or new dependencies. The new behavior is opt-in and backward compatible.
Additional context
The design was calibrated on multi-turn general-agent workloads where escalation rescued difficult phases but permanent latching dominated cost afterward.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the existing libsy escalation algorithm and its escalation configuration, then inspect the deployment-TOML, Rust, PyO3/Python, and test surfaces named in the issue. Verify that omitted options preserve the permanent latch and packaged prompt, while the opt-in policy handles minimum turns, safe-release streaks, caps, cooldowns, session state, and judge failures. Update the routing documentation and confirm the matching tests cover these behaviors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- ai, backend, documentation, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 64/100