Automatic Detection of cutoff levels for signatures

Currently cutoff levels for signatures are determined manually through trial and error and saved for future use. It would be beneficial to develop an automatic method for cutoff determination which can aid in the process of determining cutoff values.

A potentially idea for this would be the following: fit a bimodal distribution to your calculated score, then automatically calculate a 95% CI threshold above which a cell with a specific score belongs to the population with the higher score. This is defined as the cutoff value.

The method is based on the following assumptions:
- you have a bimodal distribution in your dataset
- you prefer to have false negatives than false positives

Caveats with this method:
- will not work if you only have one celltype in your dataset and thus not a bimodal distribtution

Implementation would take some work because the method is currently only implemented in R (and although running in a python Jupyter notebook requires more backend setup by the user). In addition checks need to be implemented for the above named assumptions.

This method has been applied and tested to identify naive T cells. See Example slide below.


Start by reviewing the existing R implementation and the Python Jupyter notebook setup for signature scores. Define how the bimodal distribution and 95% CI threshold should be calculated, then specify checks for the single-population and false-positive assumptions; done means automatic cutoff values can identify naive T cells without manual trial and error.

Written by the indexing model from the issue text.