ValueError: b'There are other near singularities as well. 0.090619\n'

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of scanpy.

Hi,

Trying to run scVI to analyse my data using the latest scanpy+scvi-tools workflow, as described here.

However, I'm running into a weird issue with the new seurat_v3 flavour to call HVGs. When I run this:

sc.pp.highly_variable_genes(
    adata,
    flavor = "seurat_v3",
    n_top_genes = 7000,
    layer = "counts",
    batch_key = "combined",
    subset = True
)

I get the following error:

If you pass `n_top_genes`, all cutoffs are ignored.
extracting highly variable genes
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-19-3748de5bacdc> in <module>
      5     layer = "counts",
      6     batch_key = "combined",
----> 7     subset = True
      8 )
      9 adata

/opt/conda/lib/python3.7/site-packages/scanpy/preprocessing/_highly_variable_genes.py in highly_variable_genes(adata, layer, n_top_genes, min_disp, max_disp, min_mean, max_mean, span, n_bins, flavor, subset, inplace, batch_key)
    420             span=span,
    421             subset=subset,
--> 422             inplace=inplace,
    423         )
    424 

/opt/conda/lib/python3.7/site-packages/scanpy/preprocessing/_highly_variable_genes.py in _highly_variable_genes_seurat_v3(adata, layer, n_top_genes, batch_key, span, subset, inplace)
     82         x = np.log10(mean[not_const])
     83         model = loess(x, y, span=span, degree=2)
---> 84         model.fit()
     85         estimat_var[not_const] = model.outputs.fitted_values
     86         reg_std = np.sqrt(10 ** estimat_var)

_loess.pyx in _loess.loess.fit()

ValueError: b'There are other near singularities as well. 0.090619\n'

While looking for a solution, I came across this issue that reports a similar problem.

Any ideas of what this may be?

Thanks

Versions

Reproduce the exact sc.pp.highly_variable_genes call with the reported versions, then inspect scanpy/preprocessing/_highly_variable_genes.py, especially _highly_variable_genes_seurat_v3 and its loess.fit() call. Compare the input data and dependency context with the related scvi-tools report. Done means the cause of the near-singularity error is established and a focused fix or documented limitation is identified.