scverse / scverse/scanpy

`invalid value encountered in divide` when running `pp.normalize_pearson_residuals`

Open
#2,496 6 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.6k
Forks
779
Avg merge
1d 4h
Merged PRs (30d)
27

Description

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of scanpy.
  • (optional) I have confirmed this bug exists on the master branch of scanpy.

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Hi, I'm trying to follow Analytic Pearson residuals
but after getting first 2 nomalizations, I'm stucked with Pearson residuals normalization. It seems like a devided by 0 issue? I don't know. What else I can do to check if the matrix is the problem. already did gene (min_cells=3) and cell filtering(min_genes=200).

Minimal code sample (that we can copy&paste without having any data)
analytic_pearson = sc.experimental.pp.normalize_pearson_residuals(adata, inplace=False)
/home/sxykdx/miniconda3/envs/scanpy/lib/python3.10/site-packages/scanpy/experimental/pp/_normalization.py:59: RuntimeWarning: invalid value encountered in divide
  residuals = diff / np.sqrt(mu + mu**2 / theta)

analytic_pearson["X"]
Output:

array([[-0.08038502, -0.10195383, -0.24513291, ...,  1.47699586,
        -0.08709449, -0.16926342],
       [-0.08623174, -0.109369  ,  3.53739781, ..., -0.54014355,
        -0.09342913, -0.18157157],
       [-0.07625086, -0.09671059, -0.2325321 , ..., -0.47777291,
        12.02085262,  6.06603257],
       ...,
       [-0.02799957, -0.03551299, -0.08540438, ..., -0.17560885,
        -0.03033674, -0.05896328],
       [-0.02840246, -0.03602399, -0.08663319, ..., -0.17813493,
        -0.03077326, -0.05981169],
       [-0.02914286, -0.03696307, -0.08889143, ..., -0.18277714,
        -0.03157547, -0.06137084]])
adata.layers["analytic_pearson_residuals"] = analytic_pearson["X"]
adata.layers["analytic_pearson_residuals"].sum(1)

Output:
array([nan, nan, nan, ..., nan, nan, nan])

Versions

scanpy==1.9.3 anndata==0.9.1 umap==0.5.3 numpy==1.23.5 scipy==1.10.1 pandas==2.0.1 scikit-learn==1.2.2 statsmodels==0.13.5 python-igraph==0.10.3 pynndescent==0.5.10

[Paste the output of scanpy.logging.print_versions() leaving a blank line after the details tag]

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with scanpy/experimental/pp/_normalization.py at line 59 and run the provided normalize_pearson_residuals example, then inspect analytic_pearson["X"] and its row sums. Compare the input matrix and normalization values around the division to identify the invalid case; done means the reported warning and resulting NaN row sums are addressed without changing valid residual output.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
bioinformatics, data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.