lmcinnes / lmcinnes/pynndescent

Fatal Python error: Segmentation faultFatal Python error: Segmentation fault

Open
#201 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
970
Forks
107
PR merge metrics
No merged PRs in 30d

Description

I am running the Top2Vec code on MacOS 12.6 with Python 3.8, tensorflow-macos and tensorflow-metal.
I'm getting a segmentation exception in nn_descent:
Screen Shot 2022-09-27 at 2 12 54 PM

Here's the packages:
```
% pip show pynndescent
Name: pynndescent
Version: 0.5.7
Summary: Nearest Neighbor Descent
Home-page: http://github.com/lmcinnes/pynndescent
Author: Leland McInnes
Author-email: leland.mcinnes@gmail.com
License: BSD
Location: /Users/davidlaxer/tensorflow/lib/python3.8/site-packages
Requires: joblib, llvmlite, numba, scikit-learn, scipy
Required-by: umap-learn
(tensorflow) davidlaxer@x86_64-apple-darwin13 Top2Vec % pip show umap-learn
Name: umap-learn
Version: 0.5.3
Summary: Uniform Manifold Approximation and Projection
Home-page: http://github.com/lmcinnes/umap
Author:
Author-email:
License: BSD
Location: /Users/davidlaxer/tensorflow/lib/python3.8/site-packages
Requires: numba, numpy, pynndescent, scikit-learn, scipy, tqdm
Required-by: tensorflow-similarity, top2vec

% pip show numba
Name: numba
Version: 0.56.2
Summary: compiling Python code using LLVM
Home-page: https://numba.pydata.org
Author:
Author-email:
License: BSD
Location: /Users/davidlaxer/tensorflow/lib/python3.8/site-packages
Requires: importlib-metadata, llvmlite, numpy, setuptools
Required-by: datashader, pynndescent, umap-learn

```

Here's the stack before the exception:
```
knn_search_index = NNDescent(
X,
n_neighbors=n_neighbors,
metric=metric,
metric_kwds=metric_kwds,
random_state=random_state,
n_trees=n_trees,
n_iters=n_iters,
max_candidates=60,
low_memory=low_memory,
n_jobs=n_jobs,
verbose=verbose,
compressed=False,
)

_deepcopy_atomic, copy.py:183
deepcopy, copy.py:146
_deepcopy_dict, copy.py:230
deepcopy, copy.py:146
_deepcopy_dict, copy.py:230
deepcopy, copy.py:146
_reconstruct, copy.py:270
deepcopy, copy.py:172
_deepcopy_dict, copy.py:230
deepcopy, copy.py:146
_reconstruct, copy.py:270
deepcopy, copy.py:172
_deepcopy_dict, copy.py:230
deepcopy, copy.py:146
_reconstruct, copy.py:270
deepcopy, copy.py:172
_deepcopy_dict, copy.py:230
deepcopy, copy.py:146
_deepcopy_dict, copy.py:230
deepcopy, copy.py:146
_reconstruct, copy.py:270
deepcopy, copy.py:172
slice_size, array_analysis.py:1817
to_shape, array_analysis.py:2074
_index_to_shape, array_analysis.py:2090
_analyze_op_getitem, array_analysis.py:2139
guard, ir_utils.py:1527
_analyze_expr, array_analysis.py:1571
_analyze_inst, array_analysis.py:1311
_determine_transform, array_analysis.py:1273
_run_on_blocks, array_analysis.py:1198
run, array_analysis.py:1178
_pre_run, parfor.py:2859
run, parfor.py:2867
run_pass, typed_passes.py:306
check, compiler_machinery.py:273
_runPass, compiler_machinery.py:311
_acquire_compile_lock, compiler_lock.py:35
run, compiler_machinery.py:356
_compile_core, compiler.py:486
_compile_ir, compiler.py:527
compile_ir, compiler.py:462
compile_ir, compiler.py:779
_create_gufunc_for_parfor_body, parfor_lowering.py:1509
_lower_parfor_parallel, parfor_lowering.py:316
lower_inst, lowering.py:567
lower_block, lowering.py:265
lower_function_body, lowering.py:251
lower_normal_function, lowering.py:222
lower, lowering.py:168
run_pass, typed_passes.py:394
check, compiler_machinery.py:273
_runPass, compiler_machinery.py:311
_acquire_compile_lock, compiler_lock.py:35
run, compiler_machinery.py:356
_compile_core, compiler.py:486
_compile_bytecode, compiler.py:520
compile_extra, compiler.py:452
compile_extra, compiler.py:716
_compile_core, dispatcher.py:152
_compile_cached, dispatcher.py:139
compile, dispatcher.py:125
compile, dispatcher.py:965
get_call_template, dispatcher.py:363
get_call_type, functions.py:540
_resolve_user_function_type, context.py:248
resolve_function_type, context.py:196
resolve_call, typeinfer.py:1555
resolve, typeinfer.py:601
__call__, typeinfer.py:578
propagate, typeinfer.py:155
propagate, typeinfer.py:1078
type_inference_stage, typed_passes.py:83
run_pass, typed_passes.py:105
check, compiler_machinery.py:273
_runPass, compiler_machinery.py:311
_acquire_compile_lock, compiler_lock.py:35
run, compiler_machinery.py:356
_compile_core, compiler.py:486
_compile_bytecode, compiler.py:520
compile_extra, compiler.py:452
compile_extra, compiler.py:716
_compile_core, dispatcher.py:152
_compile_cached, dispatcher.py:139
compile, dispatcher.py:125
compile, dispatcher.py:965
_compile_for_args, dispatcher.py:420
__init__, pynndescent_.py:891
nearest_neighbors, umap_.py:328
fit, umap_.py:2516
__init__, Top2Vec.py:669
, test1.py:26
```
Verbose output:
```
...
UMAP(n_neighbors=10, verbose=True)
OMP: Info #276: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
Wed Sep 28 07:46:37 2022 Construct fuzzy simplicial set
Wed Sep 28 07:46:37 2022 Finding Nearest Neighbors
Wed Sep 28 07:47:11 2022 Building RP forest with 10 trees
Wed Sep 28 07:47:12 2022 NN descent for 14 iterations
Fatal Python error: Segmentation faultFatal Python error: Segmentation fault

Thread 0x0000700011629000 (most recent call first):
File "/Users/davidlaxer/anaconda3/libFatal Python error:

/Segmentation faultp

ython3.8/threading.py", line 306 in wait
File "/Users/davidlaxer/anaconda3/lib/python3.8/threading.py", line 558 in wa
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

```
Any clues as to what's causing the exception?

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the crash from test1.py:26 with the reported macOS, Python, and package versions, then inspect pynndescent_.py:891 and umap_.py:328 around NNDescent. Done means isolating the dependency or runtime condition that causes the segmentation fault and documenting a reproducible resolution or regression coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.