block / block/buzz

Intermittent test panic in `cancelled_turn_with_usage_emits_notification_before_response`

Open
#4,945 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
32.7k
Forks
4.3k
Avg merge
1d 13h
Merged PRs (30d)
253

Description

Found while trying to reproduce https://github.com/block/buzz/issues/4939

Test `cancelled_turn_with_usage_emits_notification_before_response` fails 1-4 out of 20 times when running fake_llm tests.

**Describe the bug**

_diagnosis written by gpt-5.6-sol_

The test does:

let c_id = h.send("session/cancel", ...).await;
let _ = gate_tx.send(());

h.send() only writes the cancel request to stdin; it does not prove the agent
processed it. The test may release the gated HTTP response first. That response
contains the deliberate “no canned response” error, so session/prompt returns a
JSON-RPC error and:

v["result"]["stopReason"]

is Null.

Suggested fix: wait for the cancel acknowledgement before releasing the HTTP
gate, while retaining any usage or prompt frames received before that
acknowledgement:

let c_id = h.send("session/cancel", json!({"sessionId": sid})).await;
let (frames_before_cancel_ack, _) =
recv_until_with_drain(&mut h, |v| v["id"] == json!(c_id)).await;

let _ = gate_tx.send(());

The collected frames then need to be included in the test’s ordering assertions.

**Steps to reproduce**

```
for i in {1..20}; do
echo "attempt $i"
cargo test --release -p buzz-agent --test fake_llm
done
```

Look for

```
thread 'cancelled_turn_with_usage_emits_notification_before_response' (632197)
panicked at crates/buzz-agent/tests/fake_llm.rs:1370:13:
assertion `left == right` failed: turn must end with stopReason: cancelled
left: Null
right: "cancelled"
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

failures:
cancelled_turn_with_usage_emits_notification_before_response
```

In 20 test runs, this failure occurs 1-4 times.

**Version and platform**

buzz-desktop 0.5.5
x86_64-linux

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.