galaxyproject / galaxyproject/loom
Orbit: $7+0 < 0.05 awk filter counts NA/missing padj as significant, inflating DEG counts
- Dominant language
- TypeScript
- Stars
- 14
- Forks
- 12
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 17
Description
## What happens
During an RNA-seq differential-expression analysis, the agent extracted "significant" DEGs from a DESeq2 results table with an awk filter of the form `$7+0 < 0.05` (column 7 = `padj`). DESeq2 writes `NA` for genes filtered out by independent filtering / outlier detection. In awk, `"NA"+0` evaluates to `0`, and `0 < 0.05` is true -- so every gene with `NA` padj was silently counted as significant.
This badly inflates the DEG count. In one run, a contrast was first reported as **8,738** significant genes; re-doing the filter in Python while excluding `NA` padj gave **1,560**. A second contrast dropped to 2,627. The bad counts then propagated into downstream shared-DEG and TF analyses before the user caught it.
## Expected vs actual
- **Expected:** exclude `NA`/missing padj before applying a significance threshold, and never coerce a missing value to a number (so `NA` can't become `0`).
- **Actual:** the awk `$7+0 < 0.05` pattern treats `NA` as `0` and includes those rows as significant.
## Why this matters
This is a silent scientific-correctness failure -- the wrong numbers look completely plausible and flow into figures/tables. It's the same class of guardrail gap as #318 (count invariants in summaries) and #220 (cross-assembly gene-ID guardrail): the brain should know DESeq2 emits `NA` padj and that naive numeric coercion is unsafe.
## Suggested direction
When filtering a stats table on a p-value/padj column, the brain should prefer a method that drops non-numeric/`NA` rows explicitly (e.g. `awk '$7!="NA" && $7+0<0.05'`, or just do it in Python/R with real NA handling) rather than bare `$7+0 < 0.05`. A note in the brain guidance about DESeq2 `NA` padj + the awk coercion trap would likely prevent the whole class.
Contributor guide
No contributing guide indexed for this repository
Research direction
No file or test is named; start by locating the brain guidance or entry point that constructs stats-table filters. Confirm how DESeq2 NA padj values are handled, then document an explicit missing-value guard so NA rows cannot count as significant and verify the DEG-count workflow no longer includes them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- awk, typescript
- Domain
- bioinformatics
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100