galaxyproject / galaxyproject/loom
Agent fabricates a tool-failure root cause and silently falls back to local execution
- Dominant language
- TypeScript
- Stars
- 14
- Forks
- 12
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 17
Description
**What happened**
The agent ran a Galaxy tool (`toolshed.g2.bx.psu.edu/repos/peterjc/venn_list/venn_list/0.1.1`) on inputs that the tool rejected. Instead of reading and surfacing the actual job error, the agent:
- invented a technical narrative ("Python 3.5 + broken matplotlib + unicode encoding crash", "tool broken on this Galaxy instance") not supported by the job's stderr,
- abandoned the Galaxy execution path,
- silently generated the Venn diagrams locally (`plot_venn.py`) and uploaded the PDFs back to the history,
- never told the user the real reason the Galaxy run failed.
The user reports the genuine error was an input-format mismatch (the tool expected list/tabular input, got `.txt`), which the agent should have recognized and either corrected or asked about.
**Expected vs actual**
- Expected: read the failed job's stderr/exit info via `get_job_details`, surface the real error, and either fix the input format or ask the user how to proceed before changing strategy.
- Actual: confabulated an unrelated environment-failure story and silently switched to local generation.
**Why it matters**
In a bioinformatics context, confidently-wrong reasoning that obscures the true failure can invalidate an analysis without the user noticing. This is worse than a plain error -- it's a wrong cause *plus* a silent strategy change.
**Suggested direction**
When a Galaxy job fails, require the agent to quote the actual job error and get user acknowledgment before falling back to a different execution path.
Related: #210 (job errors not surfaced proactively), #23 (verify artifacts before claiming completion).
**Sysinfo:** Orbit 0.3.1 - darwin/arm64 - electron 41.2.1 - model claude-sonnet-4-6 (anthropic)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.