scverse / scverse/scanpy

feature selection method using pearson residuals discrepancies

Open
#3,452 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Question
Dominant language
Python
Stars
2.6k
Forks
779
Avg merge
1d 4h
Merged PRs (30d)
27

Description

Please make sure these conditions are met
  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of scanpy.
  • (optional) I have confirmed this bug exists on the main branch of scanpy.
What happened?

Hi,

I have performed multiple way to do features selection using pearson residuals and methods return different results.

Method 1 plot:
Image

Method 2 plot:
Image

I dont understand why these 2 methods dont produce the same result.

Based on those graph I think the best is method 2 but would like to understand why these dont produce similar plots.

Thanks in advance for your help

Minimal code sample
### 1st method:
analytic_pearson = sc.experimental.pp.normalize_pearson_residuals(adata, inplace=False)
adata.layers["analytic_pearson_residuals"] = csr_matrix(analytic_pearson["X"])
sc.pp.highly_variable_genes(adata, layer="analytic_pearson_residuals", n_top_genes=4000)
ax = sns.scatterplot(
data=adata.var, x="means", y="dispersions", hue="highly_variable", s=5
)

## 2nd method:

fig, ax = plt.subplots(1, 1, figsize=(10, 5))
sc.experimental.pp.highly_variable_genes(
adata, flavor="pearson_residuals", n_top_genes=4000
)
ax = sns.scatterplot(
data=adata.var, x="means", y="dispersions", hue="highly_variable", s=5
)
ax.set_xscale("log")
ax.set_yscale("log")
ax.set_title("Feature selection using Pearson residuals normalisation (from highly_variable_genes flavor pearson_residuals)")
pdf.savefig(bbox_inches="tight")


Error output

Versions

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the two workflows using normalize_pearson_residuals, highly_variable_genes with the pearson_residuals flavor, and the provided plotting code. Compare the intermediate values and selection criteria, then determine whether the differing plots are expected or indicate a defect; document the explanation or add a regression test if a defect is confirmed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
bioinformatics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.