scverse / scverse/scanpy

ValueError: b'Extrapolation not allowed with blending' when using `"sc.pp.highly_variable_genes"` function

Open
#2,853 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Needs info❔
Dominant language
Python
Stars
2.6k
Forks
779
Avg merge
1d 4h
Merged PRs (30d)
27

Description

Please make sure these conditions are met
  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of scanpy.
  • (optional) I have confirmed this bug exists on the master branch of scanpy.
What happened?

During preprocessing of concatenated adata file for scvi-based label transfer, processing fails when applying "sc.pp.highly_variable_genes" function with "ValueError: b'Extrapolation not allowed with blending'"

Minimal code sample
aadata = aadata.concatenate(ref_data_WT)

aadata.X
<15445x13343 sparse matrix of type '<class 'numpy.float64'>'
	with 107849393 stored elements in Compressed Sparse Row format>

# pre-processing:
aadata.layers["counts"] = aadata.X.copy()
sc.pp.normalize_total(aadata, target_sum=1e4)
sc.pp.log1p(aadata)
aadata.raw = aadata

sc.pp.highly_variable_genes(aadata, flavor = 'seurat_v3', n_top_genes=2000,
                            layer = "counts", batch_key="batch", subset = True)#, span =0.5
Error output
ValueError                                Traceback (most recent call last)
Cell In[37], line 7
      4 sc.pp.log1p(aadata)
      5 aadata.raw = aadata
----> 7 sc.pp.highly_variable_genes(aadata, flavor = 'seurat_v3', n_top_genes=2000,
      8                             layer = "counts", batch_key="batch", subset = True)#, span =0.5

File ~/mambaforge/envs/soupxEnv/lib/python3.10/site-packages/scanpy/preprocessing/_highly_variable_genes.py:441, in highly_variable_genes(adata, layer, n_top_genes, min_disp, max_disp, min_mean, max_mean, span, n_bins, flavor, subset, inplace, batch_key, check_values)
    439         sig = signature(_highly_variable_genes_seurat_v3)
    440         n_top_genes = cast(int, sig.parameters["n_top_genes"].default)
--> 441     return _highly_variable_genes_seurat_v3(
    442         adata,
    443         layer=layer,
    444         n_top_genes=n_top_genes,
    445         batch_key=batch_key,
    446         check_values=check_values,
    447         span=span,
    448         subset=subset,
    449         inplace=inplace,
    450     )
    452 if batch_key is None:
    453     df = _highly_variable_genes_single_batch(
    454         adata,
    455         layer=layer,
   (...)
    462         flavor=flavor,
    463     )

File ~/mambaforge/envs/soupxEnv/lib/python3.10/site-packages/scanpy/preprocessing/_highly_variable_genes.py:87, in _highly_variable_genes_seurat_v3(adata, layer, n_top_genes, batch_key, check_values, span, subset, inplace)
     85 x = np.log10(mean[not_const])
     86 model = loess(x, y, span=span, degree=2)
---> 87 model.fit()
     88 estimat_var[not_const] = model.outputs.fitted_values
     89 reg_std = np.sqrt(10**estimat_var)

File _loess.pyx:899, in _loess.loess.fit()

ValueError: b'Extrapolation not allowed with blending'
Versions
scanpy==1.9.8 anndata==0.10.2 umap==0.5.4 numpy==1.24.4 scipy==1.11.3 pandas==2.1.1 scikit-learn==1.3.1 statsmodels==0.14.0 igraph==0.10.8 pynndescent==0.5.10

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the supplied concatenated AnnData example with scanpy 1.9.8 and inspect scanpy/preprocessing/_highly_variable_genes.py, especially the seurat_v3 path and model.fit() call. Check how the loess fit reaches the “Extrapolation not allowed with blending” error, then verify that the reported batch-key and layer configuration completes and produces the requested highly variable genes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
bioinformatics
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.