NVIDIA-NeMo / NVIDIA-NeMo/Switchyard

[feature] Add optional de-escalation to the escalation router

Open
#661 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
3.2k
Forks
291
Avg merge
1d 8h
Merged PRs (30d)
182

Description

Problem

The escalation router permanently latches a session to the strong tier after escalation. In multi-turn agent workloads, the difficult part may later be resolved, but every remaining turn still uses the strong model. This raises cost and prevents the router from returning routine work to the efficient tier.

Proposed solution

Add an optional de-escalation policy under the existing escalation configuration:

  • Hold the strong tier for a configurable minimum number of turns.
  • Judge completed strong-tier responses with an explicit strong-phase marker.
  • Require consecutive safe-to-release verdicts before returning to the efficient tier.
  • Optionally cap a strong-tier run and apply an efficient-tier cooldown to prevent immediate bouncing.
  • Retain state through the existing session identity.
  • Preserve the current permanent latch and packaged prompt exactly when the option is omitted.

The policy should fail safely: judge failures keep the strong tier, and a strong-target fallback must not be judged as a strong response or preserve a partial release streak.

Alternatives considered

  • Keep permanent latching: simple, but it continues paying strong-tier cost after recovery.
  • Route every turn independently: can bounce between tiers and lose useful provider cache locality.
  • Use the stage router: useful for deterministic tool signals, but it does not replace response-based trajectory judging.

Scope notes

This belongs in the libsy escalation algorithm, with matching deployment-TOML, Rust, PyO3/Python, tests, and routing documentation. It requires no protocol changes or new dependencies. The new behavior is opt-in and backward compatible.

Additional context

The design was calibrated on multi-turn general-agent workloads where escalation rescued difficult phases but permanent latching dominated cost afterward.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the existing libsy escalation algorithm and its escalation configuration, then inspect the deployment-TOML, Rust, PyO3/Python, and test surfaces named in the issue. Verify that omitted options preserve the permanent latch and packaged prompt, while the opt-in policy handles minimum turns, safe-release streaks, caps, cooldowns, session state, and judge failures. Update the routing documentation and confirm the matching tests cover these behaviors.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
ai, backend, documentation, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.