galaxyproject / galaxyproject/loom

Brain lacks a cross-assembly gene-ID guardrail — naive locus-tag mapping produced the experiment's only scientific error

Open
#220 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
TypeScript
Stars
14
Forks
12
Avg merge
6d 5h
Merged PRs (30d)
17

Description

_Context: from a controlled experiment in which eight frontier models (opus, sonnet, gpt, gpt2, gemini-2.5-pro, gemini-3.5-flash, deepseek, haiku) each drove an Orbit session to reanalyze the same RNA-seq dataset (Santana 2023 *C. auris*) on 2026-06-02–04. Evidence quotes are from the per-run `activity.jsonl` logs._

## Problem

When the paper's gene IDs come from a different assembly version than the one analyzed, the brain tends to map locus tags **numerically** (zero-strip/zero-pad). For re-numbered assemblies this maps to the *wrong gene* and yields a false-negative biological conclusion. This was the only **scientific** error in the entire 8-model experiment.

## Evidence

- **gpt2** and **deepseek** mapped the paper's SCF1 (`B9J08_001458`, v2) to `B9J08_01458` (v3) by stripping a leading zero, then concluded SCF1 was **not differentially expressed** (gpt2 local table: `B9J08_01458` log2FC = -0.0355, padj = 0.827 — flat across all samples). Both recovered only after explicit human prompting to do protein matching.
- Correct mapping (protein reciprocal-best-hit, DIAMOND): SCF1 `B9J08_001458` → v3 `B9J08_03708`, log2FC ≈ -6.8 / -7.35, padj ≈ 0. The correct gene collapses from ~46,000 normalized counts in WT to ~400 in the mutant; the naive gene stays flat ~1,900.

## Impact

High (silent and scientific). Four of six completing runs faced this trap; two fell in and reported a wrong answer until rescued.

## Proposed direction

- Brain knowledge rule: **never** numeric-map locus tags across assembly versions; when the analysis assembly differs from the paper's, bridge IDs by protein reciprocal-best-hit (DIAMOND) or `old_locus_tag` from the GTF, and verify before interpreting.
- Pair with a `galaxy-skills` cross-assembly-ID-mapping skill (worked DIAMOND RBH recipe). Upstream, a BRC affordance that returns v2↔v3 mappings would remove the need entirely.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.