galaxyproject / galaxyproject/loom
Brain lacks a cross-assembly gene-ID guardrail — naive locus-tag mapping produced the experiment's only scientific error
- Dominant language
- TypeScript
- Stars
- 14
- Forks
- 12
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 17
Description
_Context: from a controlled experiment in which eight frontier models (opus, sonnet, gpt, gpt2, gemini-2.5-pro, gemini-3.5-flash, deepseek, haiku) each drove an Orbit session to reanalyze the same RNA-seq dataset (Santana 2023 *C. auris*) on 2026-06-02–04. Evidence quotes are from the per-run `activity.jsonl` logs._
## Problem
When the paper's gene IDs come from a different assembly version than the one analyzed, the brain tends to map locus tags **numerically** (zero-strip/zero-pad). For re-numbered assemblies this maps to the *wrong gene* and yields a false-negative biological conclusion. This was the only **scientific** error in the entire 8-model experiment.
## Evidence
- **gpt2** and **deepseek** mapped the paper's SCF1 (`B9J08_001458`, v2) to `B9J08_01458` (v3) by stripping a leading zero, then concluded SCF1 was **not differentially expressed** (gpt2 local table: `B9J08_01458` log2FC = -0.0355, padj = 0.827 — flat across all samples). Both recovered only after explicit human prompting to do protein matching.
- Correct mapping (protein reciprocal-best-hit, DIAMOND): SCF1 `B9J08_001458` → v3 `B9J08_03708`, log2FC ≈ -6.8 / -7.35, padj ≈ 0. The correct gene collapses from ~46,000 normalized counts in WT to ~400 in the mutant; the naive gene stays flat ~1,900.
## Impact
High (silent and scientific). Four of six completing runs faced this trap; two fell in and reported a wrong answer until rescued.
## Proposed direction
- Brain knowledge rule: **never** numeric-map locus tags across assembly versions; when the analysis assembly differs from the paper's, bridge IDs by protein reciprocal-best-hit (DIAMOND) or `old_locus_tag` from the GTF, and verify before interpreting.
- Pair with a `galaxy-skills` cross-assembly-ID-mapping skill (worked DIAMOND RBH recipe). Upstream, a BRC affordance that returns v2↔v3 mappings would remove the need entirely.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.