UMAP model load complication with joblib and numba
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.3k
- Forks
- 871
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 5
Description
I face a problem in loading a dumped model by joblib which is not similar to the previous raised issues, as far as I understood.
Here are the version of different relevant modules:
pathon version 3.7.7
umap version 0.4.4
pynndescent version 0.4.7
numba version 0.46.0
llvmlite version 0.30.0
And the snippet of error message:
Traceback (most recent call last):
File "", line 1, in
umap_model = joblib.load( model_dir + umap_reducer_fname )
File "C:\Users\ansa_ho\Anaconda3\lib\site-packages\joblib\numpy_pickle.py", line 585, in load
obj = _unpickle(fobj, filename, mmap_mode)
File "C:\Users\ansa_ho\Anaconda3\lib\site-packages\joblib\numpy_pickle.py", line 504, in _unpickle
obj = unpickler.load()
File "C:\Users\ansa_ho\Anaconda3\lib\pickle.py", line 1088, in load
dispatchkey[0]
File "C:\Users\ansa_ho\Anaconda3\lib\site-packages\joblib\numpy_pickle.py", line 329, in load_build
Unpickler.load_build(self)
File "C:\Users\ansa_ho\Anaconda3\lib\pickle.py", line 1552, in load_build
setstate(state)
File "C:\Users\ansa_ho\Anaconda3\lib\site-packages\pynndescent\pynndescent_.py", line 1026, in setstate
self._rp_forest = tuple([renumbaify_tree(tree) for tree in d["_rp_forest"]])
File "C:\Users\ansa_ho\Anaconda3\lib\site-packages\pynndescent\pynndescent_.py", line 1026, in
self._rp_forest = tuple([renumbaify_tree(tree) for tree in d["_rp_forest"]])
File "C:\Users\ansa_ho\Anaconda3\lib\site-packages\pynndescent\rp_trees.py", line 1178, in renumbaify_tree
hyperplanes.extend(tree.hyperplanes)
File "C:\Users\ansa_ho\Anaconda3\lib\site-packages\numba\typed\typedlist.py", line 265, in extend
return _extend(self, iterable)
File "C:\Users\ansa_ho\Anaconda3\lib\site-packages\numba\dispatcher.py", line 401, in _compile_for_args
error_rewrite(e, 'typing')
File "C:\Users\ansa_ho\Anaconda3\lib\site-packages\numba\dispatcher.py", line 344, in error_rewrite
reraise(type(e), e, None)
File "C:\Users\ansa_ho\Anaconda3\lib\site-packages\numba\six.py", line 668, in reraise
raise value.with_traceback(tb)
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Internal error at <numba.typeinfer.CallConstraint object at 0x0000023BCE0EA2C8>.
Failed in nopython mode pipeline (step: nopython mode backend)
File "......\Anaconda3\lib\site-packages\numba\listobject.py", line 434:
def impl(l, item):
casteditem = _cast(item, itemty)
^
[1] During: lowering "$0.4 = call $0.1(item, $0.3, func=$0.1, args=[Var(item, C:\Users\ansa_ho\Anaconda3\lib\site-packages\numba\listobject.py (434)), Var($0.3, C:\Users\ansa_ho\Anaconda3\lib\site-packages\numba\listobject.py (434))], kws=(), vararg=None)" at C:\Users\ansa_ho\Anaconda3\lib\site-packages\numba\listobject.py (434)
[2] During: resolving callee type: BoundFunction((<class 'numba.types.containers.ListType'>, 'append') for ListType[array(float64, 2d, C)])
[3] During: typing of call at C:\Users\ansa_ho\Anaconda3\lib\site-packages\numba\listobject.py (840)
I'd appreciate any support.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the joblib.load failure with the listed Python, UMAP, pynndescent, numba, and llvmlite versions. Then inspect pynndescent/pynndescent_.py at setstate and pynndescent/rp_trees.py at renumbaify_tree, where the traceback fails. Done means identifying a confirmed compatibility fix or a clear reproduction and resolution path for loading the dumped model.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100