jobizzness / jobizzness/bench

A crashed tab says "turn failed: success", which tells you nothing

Open
#70 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1
Forks
0
Avg merge
1h 21m
Merged PRs (30d)
41

Description

## Evidence

Two tabs crashed simultaneously during turn 76. `bench ls`:

```
test-collision sonnet crashed turn failed: success
thread-window sonnet crashed turn failed: success
```

"turn failed: success" is a contradiction on its face and carries no information about what actually went wrong. Both tabs recovered on a plain `bench tell`, which is consistent with a transient API-side failure — but nothing in the roster said so, and the developer has no way to tell that from a genuine crash they need to act on.

## Why it says that

`src/daemon/turn-outcome.ts:20`:

```ts
if (input.isError) {
return { status: "crashed", detail: `turn failed: ${input.subtype}` };
}
```

`subtype` is the SDK's result subtype, which stays `"success"` for a turn that reached its end even when `is_error` is true. The subtype answers "how did the turn terminate", not "what went wrong" — so for the most common failure it prints the least useful word available.

The text that actually describes the failure is already on the event and already typed. `src/daemon/stream-codec.ts:5-13`:

```ts
export interface ResultEvent {
type: "result";
subtype: string;
is_error: boolean;
session_id: string;
result?: string; // <- the message
...
}
```

`registry.ts:955-959` passes only `subtype` through and drops `result`.

## Acceptance criteria

- [ ] A failed turn's detail carries the SDK's `result` text when there is one, rather than the subtype
- [ ] It falls back to the subtype when `result` is absent or empty, so no case gets worse
- [ ] The detail never reads as a contradiction — no "failed: success" for any input
- [ ] Long messages are truncated to something a roster row can hold, at a stated length
- [ ] `resolveTurnOutcome` tests cover: `is_error` with a `result` string, `is_error` with no `result`, and the existing four cases still pass
- [ ] `pnpm typecheck` and `pnpm test` clean against the #50 baseline of 4 known failures

## Out of scope

- Retrying or auto-reviving a crashed tab. Saying what happened is this ticket; deciding what to do about it is not.
- Distinguishing a rate-limit crash from any other kind by parsing the message. Print what the SDK said; do not classify it.
- Anything about the `awaiting_decision` branches, which are correct.

## Verification

```
pnpm typecheck
pnpm test # 4 failures expected: dispatch-modal, model-costs, model-picker, settings-ui (#50)
```

## Related

- #65 (a concurrent test run was in flight when both crashed, but the failure was at turn level, not in the suite)

Contributor guide

Open the contributing guide

Research direction

Start with src/daemon/turn-outcome.ts and the ResultEvent definition in src/daemon/stream-codec.ts, then trace registry.ts:955-959 to see how the outcome is assembled. Run the existing resolveTurnOutcome tests and add coverage for failed results with and without result text while preserving the four existing cases. Done means the detail is useful and bounded, and pnpm typecheck plus pnpm test pass against the stated baseline.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
devtools
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
86/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.