microsoft / microsoft/amplifier

`stop_reason: "refusal"` should be clearly surfaced to the user instead of ending the turn silently

Open
#327 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
3.1k
Forks
261
Avg merge
3h 28m
Merged PRs (30d)
13

Description

Problem

When the provider (e.g., Anthropic) terminates a response with stop_reason: "refusal", Amplifier ends the turn with an empty response. From the user's perspective, the assistant "goes idle" — often right after an extended-thinking block, with no text, no tool calls, and no error message. The UX is: Amplifier appears to think, then simply stops mid-task with no explanation.

The user receives an empty response and has no signal that the turn was terminated by the provider safety layer, not by the model or any orchestrator condition.

Evidence: Real-world case from session forensics

A long-running implementation session (~27 hours, 92 LLM responses) experienced 7 turns end in stop_reason: "refusal". Every one rendered as silence:

  • Turn ends with a thinking-only assistant message (message types=thinking, no text/tool_call)
  • Then prompt:complete event with empty response
  • Thinking block content showed the model was mid-plan and intended to continue (e.g., "I'll delegate this investigation to a subagent... then check the code..." — then termination)
  • The user, seeing silence, typed "Do not stop until you are done" repeatedly — at a model that never received anything
  • Ironically, that unattended-autonomy framing likely made subsequent refusals MORE likely
  • The session idled ~9 hours overnight because a refusal killed the turn right after the user said "I'm going to sleep, do not stop until finished"
  • Even a simple "Status?" request went unanswered because that turn also ended in refusal

Context poisoning: Why refusals compound

Refusals correlate with accumulated context. Once they start, they recur. In this case the context legitimately contained:

  1. A security-incident postmortem (an AI agent that ran machine-wide taskkill/Stop-Process)
  2. Hardening work involving kill commands and silent installer execution
  3. "Run unattended overnight" instructions

Individually benign; combined, a classifier trigger profile. By the final turns, 7 of 9 turn-ends were refusals — the context became durably refusal-prone. The user had zero signal this was happening or that a fresh session was the fix.

Why this is a UX problem

  1. Silent failure: The orchestrator/CLI does not emit any signal; the user sees only "idle" and no explanation.
  2. Feedback loop: The user re-prompts the same refusal-laden context repeatedly, training themselves to add more "urgency" framing that makes refusals worse.
  3. Diagnosis requires forensics: Without session-trace analysis, there is no way to know what happened. A system message like "Provider refused to continue generating" would have saved hours.
  4. Recovery is invisible: Refusals sometimes recover mid-turn (retry/continuation is feasible), but the user never knows this is an option.

Requested improvements

  1. Surface refusals clearly in the UI/CLI: emit a message like "Provider refused to continue generating (stop_reason: refusal)" instead of silent turn end.
  2. Emit a distinct event: add a provider_refused event (or similar) to the session event stream for diagnostics.
  3. System reminder or log output: similar to loop-limit reminders, emit a notice so refusals are visible in the transcript/logs.
  4. Consider recoverable treatment: evidence shows mid-turn recovery is feasible; treat refusal as a checkpoint condition (retry/continue or notify).
  5. Documentation: add guidance that refusal-prone context accumulates; recommend fresh session or context quarantine when refusals recur (not an orchestrator limit, but a user pattern).

Suspected root cause

The orchestrator (likely loop-streaming) processes the provider response and detects stop_reason: "refusal", then ends the turn with an empty response body. The session event is logged correctly, but there is no user-facing signal — no log, no message, no system reminder.


Session reference: 507f3164-6b1e-48ff-a8ce-766e5cdbde76 (full forensic analysis available on request)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting the loop-streaming handling of provider responses and the session event stream, using session 507f3164-6b1e-48ff-a8ce-766e5cdbde76 as diagnostic context. Trace how stop_reason: "refusal" reaches prompt:complete, then define and verify a user-visible notice and distinct provider_refused event rather than an empty turn.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, cli, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.