scverse / scverse/scanpy

Bug on scanpy, sc.pp.neighbors function

Open
#2,160 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Needs info❔ Upstream Windows
Dominant language
Python
Stars
2.6k
Forks
779
Avg merge
1d 4h
Merged PRs (30d)
27

Description

Hi, I am working with a big dataset and I run into a problem when computing the neigbours. Find below an small example:

Minimal code sample

 import scanpy
import numpy
> 
tab = scvi.data.read_csv("C:/Users/RUTBO/AML-029-01-1E.gene.expression.matrix.tsv", delimiter='\t', first_column_names=True)

sc.tl.pca(tab, svd_solver='arpack')
sc.pp.neighbors(tab, n_neighbors=10, n_pcs=40)
traceback 
> The error I get:
> ---------------------------------------------------------------------------
> File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\errors.py", line 823, in new_error_context
    yield
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\lowering.py", line 293, in lower_block
    self.lower_inst(inst)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\lowering.py", line 438, in lower_inst
    val = self.lower_assign(ty, inst)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\lowering.py", line 624, in lower_assign
    return self.lower_expr(ty, value)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\lowering.py", line 1159, in lower_expr
    res = self.lower_call(resty, expr)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\lowering.py", line 889, in lower_call
    res = self._lower_call_normal(fnty, expr, signature)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\lowering.py", line 1130, in _lower_call_normal
    res = impl(self.builder, argvals, self.loc)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\base.py", line 1201, in __call__
    res = self._imp(self._context, builder, self._sig, args, loc=loc)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\base.py", line 1231, in wrapper
    return fn(*args, **kwargs)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\cpython\rangeobj.py", line 40, in range1_impl
    state.stop = stop
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\cgutils.py", line 164, in __setattr__
    self[self._datamodel.get_field_position(field)] = value
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\cgutils.py", line 188, in __setitem__
    raise TypeError("Invalid store of {value.type} to "
TypeError: Invalid store of i64 to i32 in <numba.core.datamodel.models.RangeModel object at 0x0000015592499520> (trying to write member #1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\RUTBO\PycharmProjects\pvalue\single cell analysis.py", line 44, in <module>
    sc.pp.neighbors(tab, n_neighbors=10, n_pcs=40)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\scanpy\neighbors\__init__.py", line 139, in neighbors
    neighbors.compute_neighbors(
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\scanpy\neighbors\__init__.py", line 808, in compute_neighbors
    self._distances, self._connectivities = _compute_connectivities_umap(
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\scanpy\neighbors\__init__.py", line 387, in _compute_connectivities_umap
    from umap.umap_ import fuzzy_simplicial_set
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\umap\__init__.py", line 2, in <module>
    from .umap_ import UMAP
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\umap\umap_.py", line 41, in <module>
    from umap.layouts import (
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\umap\layouts.py", line 39, in <module>
    def rdist(x, y):
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\decorators.py", line 219, in wrapper
    disp.compile(sig)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\dispatcher.py", line 965, in compile
    cres = self._compiler.compile(args, return_type)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\dispatcher.py", line 125, in compile
    status, retval = self._compile_cached(args, return_type)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\dispatcher.py", line 139, in _compile_cached
    retval = self._compile_core(args, return_type)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\dispatcher.py", line 152, in _compile_core
    cres = compiler.compile_extra(self.targetdescr.typing_context,
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\compiler.py", line 693, in compile_extra
    return pipeline.compile_extra(func)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\compiler.py", line 429, in compile_extra
    return self._compile_bytecode()
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\compiler.py", line 497, in _compile_bytecode
    return self._compile_core()
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\compiler.py", line 476, in _compile_core
    raise e
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\compiler.py", line 463, in _compile_core
    pm.run(self.state)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\compiler_machinery.py", line 353, in run
    raise patched_exception
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\compiler_machinery.py", line 341, in run
    self._runPass(idx, pass_inst, state)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\compiler_lock.py", line 35, in _acquire_compile_lock
    return func(*args, **kwargs)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\compiler_machinery.py", line 296, in _runPass
    mutated |= check(pss.run_pass, internal_state)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\compiler_machinery.py", line 269, in check
    mangled = func(compiler_state)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\typed_passes.py", line 394, in run_pass
    lower.lower()
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\lowering.py", line 196, in lower
    self.lower_normal_function(self.fndesc)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\lowering.py", line 250, in lower_normal_function
    entry_block_tail = self.lower_function_body()
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\lowering.py", line 279, in lower_function_body
    self.lower_block(block)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\lowering.py", line 293, in lower_block
    self.lower_inst(inst)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\contextlib.py", line 135, in __exit__
    self.gen.throw(type, value, traceback)
  File "C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\site-packages\numba\core\errors.py", line 837, in new_error_context
    raise newerr.with_traceback(tb)
numba.core.errors.LoweringError: Failed in nopython mode pipeline (step: native lowering)
Invalid store of i64 to i32 in <numba.core.datamodel.models.RangeModel object at 0x0000015592499520> (trying to write member #1)

File "..\..\AppData\Local\Programs\Python\Python39\lib\umap\layouts.py", line 53:
def rdist(x, y):
    <source elided>
    dim = x.shape[0]
    for i in range(dim):
    ^

During: lowering "$20call_function.7 = call $16load_global.5(dim, func=$16load_global.5, args=[Var(dim, layouts.py:52)], kws=(), vararg=None, target=None)" at C:\Users\RUTBO\AppData\Local\Programs\Python\Python39\lib\umap\layouts.py (53)

I am running scanpy using python v3.9 with numba v0.55.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the sc.pp.neighbors entry point and the traceback locations in scanpy/neighbors/init.py and umap/layouts.py. Reproduce the failure using the provided Python 3.9 and numba 0.55 context, then determine whether the actionable fix belongs in Scanpy or its dependency; done requires a verified resolution for this example.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
bioinformatics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.