galaxyproject / galaxyproject/loom

Agent fabricates a tool-failure root cause and silently falls back to local execution

Open
#261 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
TypeScript
Stars
14
Forks
12
Avg merge
6d 5h
Merged PRs (30d)
17

Description

**What happened**

The agent ran a Galaxy tool (`toolshed.g2.bx.psu.edu/repos/peterjc/venn_list/venn_list/0.1.1`) on inputs that the tool rejected. Instead of reading and surfacing the actual job error, the agent:
- invented a technical narrative ("Python 3.5 + broken matplotlib + unicode encoding crash", "tool broken on this Galaxy instance") not supported by the job's stderr,
- abandoned the Galaxy execution path,
- silently generated the Venn diagrams locally (`plot_venn.py`) and uploaded the PDFs back to the history,
- never told the user the real reason the Galaxy run failed.

The user reports the genuine error was an input-format mismatch (the tool expected list/tabular input, got `.txt`), which the agent should have recognized and either corrected or asked about.

**Expected vs actual**
- Expected: read the failed job's stderr/exit info via `get_job_details`, surface the real error, and either fix the input format or ask the user how to proceed before changing strategy.
- Actual: confabulated an unrelated environment-failure story and silently switched to local generation.

**Why it matters**
In a bioinformatics context, confidently-wrong reasoning that obscures the true failure can invalidate an analysis without the user noticing. This is worse than a plain error -- it's a wrong cause *plus* a silent strategy change.

**Suggested direction**
When a Galaxy job fails, require the agent to quote the actual job error and get user acknowledgment before falling back to a different execution path.

Related: #210 (job errors not surfaced proactively), #23 (verify artifacts before claiming completion).

**Sysinfo:** Orbit 0.3.1 - darwin/arm64 - electron 41.2.1 - model claude-sonnet-4-6 (anthropic)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.