AlexsLemonade / AlexsLemonade/refinebio

NO_OP Processor doesn't fail when it is supposed to

Open
#2,108 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
135
Forks
21
PR merge metrics
No merged PRs in 30d

Description

### Context

When we were working on creating the human compendia we ran into an issue where the processed files we were smashing together had ~500k gene identifiers in aggregate. This blew up our RAM usage. We implemented some filtering in the compendia to handle this issue, but it indicates that we have a problem with our NO_OP processor.

### Problem or idea

We may need to get some additional input from @jaclyn-taroni but I think the gist of the issue is that when the NO_OP processor performs gene conversion on submitter-processed data, it fails to check that the genes in the output file are within the geneset that we are targeting.

### Solution or next step

Add an additional step to the NO_OP processor to check the quality of the output and fail if there are too many genes that don't belong to the genset we're converting to.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the NO_OP processor and its gene-conversion path. Review how the target geneset is represented and how processed output is checked; the work is done when output with too many genes outside the target geneset causes processing to fail.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
bioinformatics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.