`invalid value encountered in divide` when running `pp.normalize_pearson_residuals`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 779
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 27
Description
- I have checked that this issue has not already been reported.
- I have confirmed this bug exists on the latest version of scanpy.
- (optional) I have confirmed this bug exists on the master branch of scanpy.
Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.
Hi, I'm trying to follow Analytic Pearson residuals
but after getting first 2 nomalizations, I'm stucked with Pearson residuals normalization. It seems like a devided by 0 issue? I don't know. What else I can do to check if the matrix is the problem. already did gene (min_cells=3) and cell filtering(min_genes=200).
Minimal code sample (that we can copy&paste without having any data)
analytic_pearson = sc.experimental.pp.normalize_pearson_residuals(adata, inplace=False)
/home/sxykdx/miniconda3/envs/scanpy/lib/python3.10/site-packages/scanpy/experimental/pp/_normalization.py:59: RuntimeWarning: invalid value encountered in divide
residuals = diff / np.sqrt(mu + mu**2 / theta)
analytic_pearson["X"]
Output:
array([[-0.08038502, -0.10195383, -0.24513291, ..., 1.47699586,
-0.08709449, -0.16926342],
[-0.08623174, -0.109369 , 3.53739781, ..., -0.54014355,
-0.09342913, -0.18157157],
[-0.07625086, -0.09671059, -0.2325321 , ..., -0.47777291,
12.02085262, 6.06603257],
...,
[-0.02799957, -0.03551299, -0.08540438, ..., -0.17560885,
-0.03033674, -0.05896328],
[-0.02840246, -0.03602399, -0.08663319, ..., -0.17813493,
-0.03077326, -0.05981169],
[-0.02914286, -0.03696307, -0.08889143, ..., -0.18277714,
-0.03157547, -0.06137084]])
adata.layers["analytic_pearson_residuals"] = analytic_pearson["X"]
adata.layers["analytic_pearson_residuals"].sum(1)
Output:
array([nan, nan, nan, ..., nan, nan, nan])
Versions
scanpy==1.9.3 anndata==0.9.1 umap==0.5.3 numpy==1.23.5 scipy==1.10.1 pandas==2.0.1 scikit-learn==1.2.2 statsmodels==0.13.5 python-igraph==0.10.3 pynndescent==0.5.10
[Paste the output of scanpy.logging.print_versions() leaving a blank line after the details tag]
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with scanpy/experimental/pp/_normalization.py at line 59 and run the provided normalize_pearson_residuals example, then inspect analytic_pearson["X"] and its row sums. Compare the input matrix and normalization values around the division to identify the invalid case; done means the reported warning and resulting NaN row sums are addressed without changing valid residual output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- bioinformatics, data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100