All NaN for an example from document
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.3k
- Forks
- 871
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 5
Description
I tried to run the prime number example in https://umap-learn.readthedocs.io/en/latest/sparse.html. However it got a lot NaN's both for my local pc and google colab.
versions:
Requirement already satisfied: umap-learn[plot] in /root/.local/lib/python3.7/site-packages (0.5.1)
Requirement already satisfied: scipy>=1.0 in /usr/local/lib/python3.7/dist-packages (from umap-learn[plot]) (1.4.1)
Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from umap-learn[plot]) (1.19.5)
Requirement already satisfied: pynndescent>=0.5 in /root/.local/lib/python3.7/site-packages (from umap-learn[plot]) (0.5.2)
Requirement already satisfied: numba>=0.49 in /usr/local/lib/python3.7/dist-packages (from umap-learn[plot]) (0.51.2)
Requirement already satisfied: scikit-learn>=0.22 in /usr/local/lib/python3.7/dist-packages (from umap-learn[plot]) (0.22.2.post1)
Requirement already satisfied: bokeh; extra == "plot" in /usr/local/lib/python3.7/dist-packages (from umap-learn[plot]) (2.3.2)
Requirement already satisfied: matplotlib; extra == "plot" in /usr/local/lib/python3.7/dist-packages (from umap-learn[plot]) (3.2.2)
Requirement already satisfied: scikit-image; extra == "plot" in /usr/local/lib/python3.7/dist-packages (from umap-learn[plot]) (0.16.2)
Requirement already satisfied: seaborn; extra == "plot" in /usr/local/lib/python3.7/dist-packages (from umap-learn[plot]) (0.11.1)
Requirement already satisfied: holoviews; extra == "plot" in /usr/local/lib/python3.7/dist-packages (from umap-learn[plot]) (1.14.4)
Requirement already satisfied: pandas; extra == "plot" in /usr/local/lib/python3.7/dist-packages (from umap-learn[plot]) (1.1.5)
Requirement already satisfied: colorcet; extra == "plot" in /usr/local/lib/python3.7/dist-packages (from umap-learn[plot]) (2.0.6)
Requirement already satisfied: datashader; extra == "plot" in /root/.local/lib/python3.7/site-packages (from umap-learn[plot]) (0.13.0)
Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from pynndescent>=0.5->umap-learn[plot]) (1.0.1)
Requirement already satisfied: llvmlite>=0.30 in /usr/local/lib/python3.7/dist-packages (from pynndescent>=0.5->umap-learn[plot]) (0.34.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from numba>=0.49->umap-learn[plot]) (57.0.0)
Requirement already satisfied: tornado>=5.1 in /usr/local/lib/python3.7/dist-packages (from bokeh; extra == "plot"->umap-learn[plot]) (5.1.1)
Requirement already satisfied: pillow>=7.1.0 in /usr/local/lib/python3.7/dist-packages (from bokeh; extra == "plot"->umap-learn[plot]) (7.1.2)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from bokeh; extra == "plot"->umap-learn[plot]) (2.8.1)
Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from bokeh; extra == "plot"->umap-learn[plot]) (3.7.4.3)
Requirement already satisfied: Jinja2>=2.9 in /usr/local/lib/python3.7/dist-packages (from bokeh; extra == "plot"->umap-learn[plot]) (2.11.3)
Requirement already satisfied: packaging>=16.8 in /usr/local/lib/python3.7/dist-packages (from bokeh; extra == "plot"->umap-learn[plot]) (20.9)
Requirement already satisfied: PyYAML>=3.10 in /usr/local/lib/python3.7/dist-packages (from bokeh; extra == "plot"->umap-learn[plot]) (3.13)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib; extra == "plot"->umap-learn[plot]) (0.10.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib; extra == "plot"->umap-learn[plot]) (2.4.7)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib; extra == "plot"->umap-learn[plot]) (1.3.1)
Requirement already satisfied: PyWavelets>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image; extra == "plot"->umap-learn[plot]) (1.1.1)
Requirement already satisfied: networkx>=2.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image; extra == "plot"->umap-learn[plot]) (2.5.1)
Requirement already satisfied: imageio>=2.3.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image; extra == "plot"->umap-learn[plot]) (2.4.1)
Requirement already satisfied: panel>=0.8.0 in /usr/local/lib/python3.7/dist-packages (from holoviews; extra == "plot"->umap-learn[plot]) (0.11.3)
Requirement already satisfied: pyviz-comms>=0.7.4 in /usr/local/lib/python3.7/dist-packages (from holoviews; extra == "plot"->umap-learn[plot]) (2.0.2)
Requirement already satisfied: param<2.0,>=1.9.3 in /usr/local/lib/python3.7/dist-packages (from holoviews; extra == "plot"->umap-learn[plot]) (1.10.1)
Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas; extra == "plot"->umap-learn[plot]) (2018.9)
Requirement already satisfied: pyct>=0.4.4 in /usr/local/lib/python3.7/dist-packages (from colorcet; extra == "plot"->umap-learn[plot]) (0.4.8)
Requirement already satisfied: xarray>=0.9.6 in /usr/local/lib/python3.7/dist-packages (from datashader; extra == "plot"->umap-learn[plot]) (0.18.2)
Requirement already satisfied: datashape>=0.5.1 in /root/.local/lib/python3.7/site-packages (from datashader; extra == "plot"->umap-learn[plot]) (0.5.2)
Requirement already satisfied: dask[complete]>=0.18.0 in /usr/local/lib/python3.7/dist-packages (from datashader; extra == "plot"->umap-learn[plot]) (2.12.0)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->bokeh; extra == "plot"->umap-learn[plot]) (1.15.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from Jinja2>=2.9->bokeh; extra == "plot"->umap-learn[plot]) (2.0.1)
Requirement already satisfied: decorator<5,>=4.3 in /usr/local/lib/python3.7/dist-packages (from networkx>=2.0->scikit-image; extra == "plot"->umap-learn[plot]) (4.4.2)
Requirement already satisfied: markdown in /usr/local/lib/python3.7/dist-packages (from panel>=0.8.0->holoviews; extra == "plot"->umap-learn[plot]) (3.3.4)
Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from panel>=0.8.0->holoviews; extra == "plot"->umap-learn[plot]) (4.41.1)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from panel>=0.8.0->holoviews; extra == "plot"->umap-learn[plot]) (2.23.0)
Requirement already satisfied: multipledispatch>=0.4.7 in /root/.local/lib/python3.7/site-packages (from datashape>=0.5.1->datashader; extra == "plot"->umap-learn[plot]) (0.6.0)
Requirement already satisfied: partd>=0.3.10; extra == "complete" in /root/.local/lib/python3.7/site-packages (from dask[complete]>=0.18.0->datashader; extra == "plot"->umap-learn[plot]) (1.2.0)
Requirement already satisfied: toolz>=0.7.3; extra == "complete" in /usr/local/lib/python3.7/dist-packages (from dask[complete]>=0.18.0->datashader; extra == "plot"->umap-learn[plot]) (0.11.1)
Requirement already satisfied: fsspec>=0.6.0; extra == "complete" in /root/.local/lib/python3.7/site-packages (from dask[complete]>=0.18.0->datashader; extra == "plot"->umap-learn[plot]) (2021.6.0)
Requirement already satisfied: distributed>=2.0; extra == "complete" in /root/.local/lib/python3.7/site-packages (from dask[complete]>=0.18.0->datashader; extra == "plot"->umap-learn[plot]) (2021.6.0)
Requirement already satisfied: cloudpickle>=0.2.1; extra == "complete" in /usr/local/lib/python3.7/dist-packages (from dask[complete]>=0.18.0->datashader; extra == "plot"->umap-learn[plot]) (1.3.0)
Requirement already satisfied: importlib-metadata; python_version < "3.8" in /usr/local/lib/python3.7/dist-packages (from markdown->panel>=0.8.0->holoviews; extra == "plot"->umap-learn[plot]) (4.5.0)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->panel>=0.8.0->holoviews; extra == "plot"->umap-learn[plot]) (2.10)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->panel>=0.8.0->holoviews; extra == "plot"->umap-learn[plot]) (2021.5.30)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->panel>=0.8.0->holoviews; extra == "plot"->umap-learn[plot]) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->panel>=0.8.0->holoviews; extra == "plot"->umap-learn[plot]) (1.24.3)
Requirement already satisfied: locket in /root/.local/lib/python3.7/site-packages (from partd>=0.3.10; extra == "complete"->dask[complete]>=0.18.0->datashader; extra == "plot"->umap-learn[plot]) (0.2.1)
Requirement already satisfied: zict>=0.1.3 in /usr/local/lib/python3.7/dist-packages (from distributed>=2.0; extra == "complete"->dask[complete]>=0.18.0->datashader; extra == "plot"->umap-learn[plot]) (2.0.0)
Requirement already satisfied: psutil>=5.0 in /usr/local/lib/python3.7/dist-packages (from distributed>=2.0; extra == "complete"->dask[complete]>=0.18.0->datashader; extra == "plot"->umap-learn[plot]) (5.4.8)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from distributed>=2.0; extra == "complete"->dask[complete]>=0.18.0->datashader; extra == "plot"->umap-learn[plot]) (1.0.2)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.7/dist-packages (from distributed>=2.0; extra == "complete"->dask[complete]>=0.18.0->datashader; extra == "plot"->umap-learn[plot]) (7.1.2)
Requirement already satisfied: tblib>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from distributed>=2.0; extra == "complete"->dask[complete]>=0.18.0->datashader; extra == "plot"->umap-learn[plot]) (1.7.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /usr/local/lib/python3.7/dist-packages (from distributed>=2.0; extra == "complete"->dask[complete]>=0.18.0->datashader; extra == "plot"->umap-learn[plot]) (2.4.0)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata; python_version < "3.8"->markdown->panel>=0.8.0->holoviews; extra == "plot"->umap-learn[plot]) (3.4.1)
Requirement already satisfied: heapdict in /usr/local/lib/python3.7/dist-packages (from zict>=0.1.3->distributed>=2.0; extra == "complete"->dask[complete]>=0.18.0->datashader; extra == "plot"->umap-learn[plot]) (1.0.1)
Codes:
import numpy as np
import scipy.sparse
import sympy
import sklearn.datasets
import sklearn.feature_extraction.text
import umap
import umap.plot
import matplotlib.pyplot as plt
%matplotlib inline
primes = list(sympy.primerange(2, 110000))
prime_to_column = {p:i for i, p in enumerate(primes)}
%%time
lil_matrix_rows = []
lil_matrix_data = []
for n in range(100000):
prime_factors = sympy.primefactors(n)
lil_matrix_rows.append([prime_to_column[p] for p in prime_factors])
lil_matrix_data.append([1] * len(prime_factors))
factor_matrix = scipy.sparse.lil_matrix((len(lil_matrix_rows), len(primes)), dtype=np.float32)
factor_matrix.rows = np.array(lil_matrix_rows)
factor_matrix.data = np.array(lil_matrix_data)
mapper = umap.UMAP(metric='cosine', random_state=42, low_memory=True).fit(factor_matrix)
umap.plot.points(mapper, values=np.arange(100000), theme='viridis')
y=mapper.transform(factor_matrix)
Most values in y are NaN's (189874 of 200000).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the prime-factor example from sparse.html with the reported package versions, then inspect the outputs of fit, plot.points, and transform on factor_matrix. Done means identifying why transform produces NaN values and either correcting the behavior or documenting the supported conditions and expected result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python, scikit-learn
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100