AlexsLemonade / AlexsLemonade/sc-data-integration

Permute celldex ref labels and compare true cell assignment to distribution of assignments

Open
#208 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

We will need some way to measure the confidence in the label assignment obtained from running `SingleR` with a given reference. One way of doing this would be to shuffle the sample labels of the reference dataset prior to training/ identifying marker genes and classifying cell types in the test dataset. This should be done over a set number of permutations to obtain a distribution of cell type assignments for each cell in the test dataset. We can then compare the true label to the distribution of assignments to obtain a p-value.

Before we can do this we will need to figure out the following:
- The run time for training and classifying to see an estimate of how long we think this will take.
- Figure out either a score or some sort of output value that is comparable across runs of `SingleR` to use for comparing the true label to the distribution. For example, if we can use the score computed by `SingleR`, we would compare the score for a given cell corresponding to the true label to the score for that cell for that same true label across each permutation.

We will probably want to have a function that takes as input the sce object of interest and the reference data to be used. Then within that function, the permutations will be performed prior to running `SingleR`. We should also use parallelization whenever possible.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by benchmarking the runtime of training and classifying with SingleR, then determine whether its scores are comparable across runs. Design a function accepting the SCE object and reference data, performing repeated label permutations before SingleR classification, with parallelization where possible. Done means the true-label assignment can be compared with the permutation distribution to obtain a p-value.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
bioinformatics
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.