MCPJam / MCPJam/inspector

Swarms: rubric findings can never link to the session they failed on

Open
#5,190 1 comment 0 reactions 0 assignees View on GitHub
bug size:S
Dominant language
TypeScript
Stars
2.2k
Forks
283
Avg merge
11h 47m
Merged PRs (30d)
737

Description

Split out of #5184. This is the half of the original report that was never about the outage — it is broken on a healthy run too.

## What happens

In the journey diagnostic, a failed rubric renders as text with no way to reach the session it is about:

```
Rubric check "Final message non-empty" failed
Session 1 (no preview)
```

The original report asked why it shows "that Rubric check language rather than just linking to sessions". The answer is that the link cannot render, by construction.

## Cause

`client/src/components/swarms/findings/findings-derivation.ts:198`:

```ts
function rubricEvidence(run: SwarmOverviewRun): StageEvidence[] {
return run.findings.map((finding) => ({
tone: findingSeverity(finding) === "blocking" ? "fail" : ("warn" as const),
observation: `Rubric check "${findingName(finding)}" failed`,
meta: findingSessionLabel(finding),
}));
}
```

No `sessionId`. Only `detectorEvidence` emits one (from `candidate.exemplarSessionIds[0]`), so the "Open source session →" affordance in `findings-goal-inspect.tsx` can never render for a rubric row.

`run.findings` comes from `getSwarmOverview`, which by design reads run rows only — the stored `criterionSummary` — and never touches sessions, so the exemplar ids are not available at that layer.

## Also

The expandable session list underneath *is* clickable, but `FindingsGoalSessions` calls `onOpenSession(session._id)` with one argument while `handleOpenSession` takes `(sessionId, criterionId)`. So `criterionId` is always `undefined` and the "Following finding:" banner never appears when arriving from there. Small, same area.

## Fix

Either carry exemplar session ids into the finding DTO from `getSwarmOverview` (which changes what that query is willing to read), or have the rubric row resolve its sessions through the drilldown the expandable list already uses — `useGoalOutcomeDrilldown` is correctly run-scoped and already returns the rows.

The second is probably the smaller change and keeps `getSwarmOverview`'s contract intact.

Contributor guide

Open the contributing guide

Research direction

Start in client/src/components/swarms/findings/findings-derivation.ts at rubricEvidence, then read findings-goal-inspect.tsx, FindingsGoalSessions, handleOpenSession, and useGoalOutcomeDrilldown. Confirm how the expandable session list resolves its rows. Done means failed rubric rows can open their source session and navigation from the session list preserves the criterion so the "Following finding:" banner appears.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
frontend, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.