microsoft / microsoft/amplifier

[provider-anthropic] fallback_cooldown_seconds: 0 reports a fallback window it never opened, masking the overload error

Open
#386 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.1k
Forks
261
Avg merge
3h 28m
Merged PRs (30d)
13

Description

fallback_cooldown_seconds accepts 0, but at 0 the overload-fallback path reports success without doing anything, and the caller then raises a RuntimeError that replaces the real overload error.

Verified against microsoft/amplifier-module-provider-anthropic@833403b (current main). Not reproduced live — this is read from the control flow.

The path

  1. fallback_cooldown_seconds: 0 survives config load. The clamp is max(0.0, ...), so 0 is a legal value, not a rejected one:

    self._fallback_cooldown_seconds = max(
        0.0, self._config_float(self.config.get("fallback_cooldown_seconds", 300.0), 300.0)
    )
    
  2. An overload error reaches _open_fallback_window. It builds the window, then stores it only when the cooldown is positive — but returns True unconditionally:

    if self._fallback_cooldown_seconds > 0:
        _set_fallback_window(family, window)
        self._write_shared_fallback_state(family, window)
    ...
    return True
    
  3. The caller in complete() reads True as "a downgrade window is open, go around again":

    if not await self._open_fallback_window(effective_model, e):
        raise
    
  4. Next iteration, _resolve_effective_model finds no window and returns the same model. It is already in attempted_models, and full_retry_budget_used was not touched (that branch requires not self._is_overload_fallback_error(e)), so the loop guard fires:

    raise RuntimeError(f"Overload fallback loop detected while resolving {requested_model}")
    

The operator set a config key to zero and got Overload fallback loop detected instead of the overload error. The API error that actually caused it is gone.

Suggested fix, roughly in order of preference:

  • Return False when self._fallback_cooldown_seconds <= 0 — no window was opened, so the caller should re-raise the original error, which is the correct outcome.
  • Or reject 0 at config load with a warning and fall back to the 300s default, if a zero cooldown is not meant to be a way of disabling fallback.
  • Or, if 0 is meant to mean "downgrade once, no persistent window", make that explicit — the current code does not implement it either way.

Found while reviewing this provider against a local fork; the fork does not carry a fix for this, since the fix depends on which of the three readings is intended.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in the provider-anthropic control flow around config loading, _open_fallback_window, complete(), and _resolve_effective_model. Confirm how fallback_cooldown_seconds=0 is intended to behave before choosing among the proposed semantics. Done means the zero-value path no longer masks the original overload error, with coverage for that control flow.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.