galaxyproject / galaxyproject/loom
Orbit: agent answers "why did my job fail?" by resuming an inaccessible history (403) and hallucinating a failure explanation
- Dominant language
- TypeScript
- Stars
- 14
- Forks
- 12
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 17
Description
**App:** 0.5.1 (linux) · **Model:** kimi-k2.7-code (openai-compatible). Reported via beta feedback ("Total hallucination — the history does not exist").
## What happened
The user asked *"why my last galaxy job failed?"* The agent worked off the notebook-pages list instead of the user's own jobs:
1. `notebook_list_galaxy_pages` → returned a set of Galaxy pages
2. `notebook_resume_from_galaxy(page_id=…, history_id=…)` → linked to a history taken from one of those pages
3. `galaxy_history_show` → **tool not found** (the agent guessed a tool name that doesn't exist on the server)
4. `galaxy_get_history_contents(history_id=…)` → **403 `History is not accessible by user` (err_code 403002)** — that history isn't owned by the current user/API key
After the 403 it didn't surface the access error; it fabricated an answer about the inaccessible history — hence the reporter's "Total hallucination."
## The bugs
1. **Wrong data source for "my last job."** "Why did my last job fail?" should resolve against the user's *own* histories/jobs (`get_histories` / `list_history_ids` → recent jobs), not a history pulled from a notebook page that may belong to someone else.
2. **403 → confident hallucination.** A 403 / not-accessible (or a guessed tool that 404s) should be surfaced as "I couldn't access that history," not narrated around.
## Suggested direction
- Ground "my last job/history" queries in the user's accessible histories before answering.
- Treat a Galaxy 403 / not-accessible as a hard stop that's reported to the user, not something to invent a narrative around.
xref #261 (agent fabricates a tool-failure root cause), #210 (job errors not surfaced proactively).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the agent handling for “my last job” alongside notebook_list_galaxy_pages, notebook_resume_from_galaxy, get_histories, and list_history_ids. Check how galaxy_get_history_contents returns a 403 and how unknown tools are handled; done means accessible user histories are preferred and inaccessible or unavailable data is reported without an explanation being generated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100