anthropics / anthropics/claude-code

`/ultrareview` results become permanently unrecoverable once the task ID expires

Aperta
#88,606 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
area:cost area:skills bug has repro
Lingua principale
Python
Stelle
145k
Fork
23.1k
Metriche di merge delle PR
Metriche PR in attesa

Descrizione

# `/ultrareview` results become permanently unrecoverable once the task ID expires

## Summary

`/ultrareview` is a paid/quota-limited feature (e.g. "Free ultrareview 2 of 3") that runs an expensive cloud review. Once it completes, its findings are delivered as a `` into the *assistant's* model context only — the user never sees this raw payload directly; the user's only view of the results is the assistant's own prose reproduction of them in the chat. There is no user-facing, durable way to retrieve the raw output at all: the tracking URL shows nothing, and `TaskOutput` stops resolving the task ID well within the same session (so even the assistant can't re-fetch it). The only way to get a review's data again is to re-run `/ultrareview`, which consumes another one of the limited runs and produces a fresh (not identical) analysis.

Because this feature specifically costs real quota per invocation, having no user-facing access to its raw output — only a model-mediated retelling — is a bigger problem than it would be for a free/cheap background task. You can't just "run it again" for free to recover the data, and you can't get the actual results without trusting the assistant's regeneration of them.

## Repro

1. Ran `/ultrareview` on a branch (billed against a limited quota — "Free ultrareview 2 of 3"). It launched a cloud review and returned a task ID plus a tracking URL (`https://claude.ai/code/session_...`).
2. The review completed; a `` delivered the full JSON findings payload into the assistant's context — not anywhere the user could see it directly. The user's only visibility into the results was the assistant retyping/summarizing them in chat.
3. User visited the tracking URL afterward — no output was shown there, so there was no user-facing way to see the raw results either.
4. Later in the same session, the assistant called `TaskOutput` with the same task ID to try to re-fetch the raw payload on the user's behalf:
```
TaskOutput({ task_id: "rccpjvyqu", block: false })
→ Error: No task found with ID: rccpjvyqu
```

## Expected

Given `/ultrareview` charges against a limited quota per run, the user should have some direct, durable, user-facing way to see the actual results — not only the assistant's paraphrase of them. Either:
- the tracking URL should actually show the completed review's raw output to the user, or
- the raw findings should be surfaced to the user directly in the conversation (e.g. as a file, artifact, or rendered block) at delivery time, not only handed to the assistant's context, or
- `TaskOutput` (or an equivalent) should still retrieve the completed review's result for a reasonable retention window so it can be recovered later, or
- the task notification should include a stable output file/link that persists independently of the ephemeral task ID.

## Actual

- The user never had direct access to the raw findings at all — only the assistant's own chat reproduction of them.
- The tracking URL showed nothing.
- The task ID was already unresolvable via `TaskOutput` well within the same conversation/session, so not even the assistant could recover the original data afterward.

## Impact

For a feature that costs real, limited quota per run, users can't get a guaranteed-exact copy of the findings at all — they only ever see the assistant's regenerated retelling of them, with no fallback to the actual source once the task ID expires. Re-running the review to "get the data" burns another one of a limited number of paid/quota-gated runs.

This also locks in whichever model happened to be driving the session at delivery time. In this case the session was running Sonnet, so the retelling of an expensive, quota-gated review is permanently capped at Sonnet-quality synthesis — there's no way to later have a higher-tier model (Opus, Fable) re-read the same raw findings and produce a better analysis, because the raw data no longer exists anywhere retrievable. The model tier active when the task notification lands becomes a permanent, unchangeable property of the result.

## Related

- #87285 — a different failure mode on the same theme: there, the `task-notification` never arrives at all and the browser session 403s. Here, the notification *does* arrive (into the assistant's context) but still can't be recovered afterward by anyone.
- #63331 — docs describe a `/tasks` command and `claude ultrareview --json` subcommand for reading findings after the fact; neither exists in the shipped CLI.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

Start by tracing the /ultrareview flow, task-notification delivery, TaskOutput lookup, and tracking URL behavior described in the report. Done means completed review findings remain directly and durably accessible to the user without rerunning the quota-limited review or relying only on an assistant paraphrase.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
cli, cloud
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Attiva
Chiarezza
Abbastanza chiara
Idoneità per principianti
50/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.