blekhmanlab / blekhmanlab/compendium_website

Subsample user input samples

Open
#34 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
2
Forks
1
PR merge metrics
No merged PRs in 30d

Description

I just realized there's another another relatively unusual operation we'll need to figure out for the projectionist app: In the user's input table, each row is a sample, and the sum of each row is the total read count for that sample. It's almost always less than 100,000 reads per sample, but a few big projects end up having 1 million+. Here's what we found in the original compendium dataset:

Image

To minimize the effect of some samples/projects having way more reads, we need to *rarefy* each sample down to 3000 reads. Starting with a pool of, say, 100,000 reads in a microbiome sample, we need to select a random subset of 3000 reads (without replacement). Unfortunately, I'm not aware of a specific tool for doing this when the items are counted already, rather than just living in an array of length 10,000.

In case I've managed to make this even more confusing, here's a sketch of what it might look like:

**Original data**

| Sample | Tax 1 | Tax 2 | Tax 3 | Tax 4 | Tax 5 | READ TOTAL |
| ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- |
| s123 | 400 | 200 | 1000 | 1100 | 1900 | 4600 |
| s124 | 1000 | 5000 | 8400 | 200 | 10 | 14610 |
| s125 | 0 | 1 | 8000 | 7000 | 1400 | 16401 |

**Subsampled**

| Sample | Tax 1 | Tax 2 | Tax 3 | Tax 4 | Tax 5 | READ TOTAL |
| ------------- | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- |
| s123 | 300 | 150 | 600 | 650 | 1300 | 3000 |
| s124 | 250 | 1000 | 1700 | 50 | 0 | 3000 |
| s125 | 0 | 0 | 1400 | 1300 | 300 | 3000 |

These subsampled values would be the ones that are then CLR-transformed.

For what it's worth, this step has an R implementation in [vegan](https://www.rdocumentation.org/packages/vegan/versions/2.7-2/topics/rarefy) and will be implemented in microbiomap to generate production-quality figures. If it would help to set some kind of limit in the browser—rejecting files with any samples with >250,000 reads, say—it would be trivial to add a helper function that takes the user input file and generates a new, rarefied one.

Contributor guide

No contributing guide indexed for this repository

Research direction

No source files, tests, or entry points are named. Start by reviewing the issue's vegan rarefy reference and decide whether the browser should perform random subsampling or reject oversized inputs; done means the chosen scope produces or enforces 3000-read samples before CLR transformation.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
data, web-dev
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.