scikit-learn / scikit-learn/scikit-learn
test_uniform_grid[barnes_hut] can fail on aarch64
Open
Nobody has claimed this yet.
arch:arm
Bug
help wanted
module:test-suite
- Dominant language
- Python
- Stars
- 67.3k
- Forks
- 27.4k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 58
Description
________________________ test_uniform_grid[barnes_hut] _________________________
method = 'barnes_hut'
@pytest.mark.parametrize('method', ['barnes_hut', 'exact'])
def test_uniform_grid(method):
"""Make sure that TSNE can approximately recover a uniform 2D grid
Due to ties in distances between point in X_2d_grid, this test is platform
dependent for ``method='barnes_hut'`` due to numerical imprecision.
Also, t-SNE is not assured to converge to the right solution because bad
initialization can lead to convergence to bad local minimum (the
optimization problem is non-convex). To avoid breaking the test too often,
we re-run t-SNE from the final point when the convergence is not good
enough.
"""
seeds = [0, 1, 2]
n_iter = 500
for seed in seeds:
tsne = TSNE(n_components=2, init='random', random_state=seed,
perplexity=20, n_iter=n_iter, method=method)
Y = tsne.fit_transform(X_2d_grid)
try_name = "{}_{}".format(method, seed)
try:
> assert_uniform_grid(Y, try_name)
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.8/site-packages/sklearn/manifold/tests/test_t_sne.py:784:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Y = array([[ 52.326397 , -15.92225 ],
[ 46.679527 , -20.175953 ],
[ 40.870537 , -24.181147 ],
...[-35.291374 , 22.122814 ],
[-42.2738 , 18.793724 ],
[-48.922283 , 15.606232 ]], dtype=float32)
try_name = 'barnes_hut_1'
def assert_uniform_grid(Y, try_name=None):
# Ensure that the resulting embedding leads to approximately
# uniformly spaced points: the distance to the closest neighbors
# should be non-zero and approximately constant.
nn = NearestNeighbors(n_neighbors=1).fit(Y)
dist_to_nn = nn.kneighbors(return_distance=True)[0].ravel()
assert dist_to_nn.min() > 0.1
smallest_to_mean = dist_to_nn.min() / np.mean(dist_to_nn)
largest_to_mean = dist_to_nn.max() / np.mean(dist_to_nn)
assert smallest_to_mean > .5, try_name
> assert largest_to_mean < 2, try_name
E AssertionError: barnes_hut_1
E assert 6.67359409617653 < 2
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.8/site-packages/sklearn/manifold/tests/test_t_sne.py:807: AssertionError
During handling of the above exception, another exception occurred:
method = 'barnes_hut'
@pytest.mark.parametrize('method', ['barnes_hut', 'exact'])
def test_uniform_grid(method):
"""Make sure that TSNE can approximately recover a uniform 2D grid
Due to ties in distances between point in X_2d_grid, this test is platform
dependent for ``method='barnes_hut'`` due to numerical imprecision.
Also, t-SNE is not assured to converge to the right solution because bad
initialization can lead to convergence to bad local minimum (the
optimization problem is non-convex). To avoid breaking the test too often,
we re-run t-SNE from the final point when the convergence is not good
enough.
"""
seeds = [0, 1, 2]
n_iter = 500
for seed in seeds:
tsne = TSNE(n_components=2, init='random', random_state=seed,
perplexity=20, n_iter=n_iter, method=method)
Y = tsne.fit_transform(X_2d_grid)
try_name = "{}_{}".format(method, seed)
try:
assert_uniform_grid(Y, try_name)
except AssertionError:
# If the test fails a first time, re-run with init=Y to see if
# this was caused by a bad initialization. Note that this will
# also run an early_exaggeration step.
try_name += ":rerun"
tsne.init = Y
Y = tsne.fit_transform(X_2d_grid)
> assert_uniform_grid(Y, try_name)
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.8/site-packages/sklearn/manifold/tests/test_t_sne.py:792:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Y = array([[-18.169476 , 6.0802336 ],
[-18.278513 , 2.8822129 ],
[-18.671782 , -0.4646889 ],
...[ 22.550077 , 19.698557 ],
[ 21.399723 , 22.933178 ],
[ 16.22136 , 28.22955 ]], dtype=float32)
try_name = 'barnes_hut_1:rerun'
def assert_uniform_grid(Y, try_name=None):
# Ensure that the resulting embedding leads to approximately
# uniformly spaced points: the distance to the closest neighbors
# should be non-zero and approximately constant.
nn = NearestNeighbors(n_neighbors=1).fit(Y)
dist_to_nn = nn.kneighbors(return_distance=True)[0].ravel()
assert dist_to_nn.min() > 0.1
smallest_to_mean = dist_to_nn.min() / np.mean(dist_to_nn)
largest_to_mean = dist_to_nn.max() / np.mean(dist_to_nn)
assert smallest_to_mean > .5, try_name
> assert largest_to_mean < 2, try_name
E AssertionError: barnes_hut_1:rerun
E assert 2.145051767903112 < 2
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/python3.8/site-packages/sklearn/manifold/tests/test_t_sne.py:807: AssertionError
The following NEW packages will be INSTALLED:
apipkg: 1.5-py_0 conda-forge
attrs: 19.3.0-py_0 conda-forge
binutils_impl_linux-aarch64: 2.29.1-hc862510_0 c4aarch64
ca-certificates: 2019.11.28-hecc5488_0 conda-forge
certifi: 2019.11.28-py38_0 conda-forge
cython: 0.29.14-py38he1b5a44_0 conda-forge
execnet: 1.7.1-py_0 conda-forge
importlib_metadata: 1.2.0-py38_0 conda-forge
joblib: 0.14.0-py_0 conda-forge
libblas: 3.8.0-14_openblas conda-forge
libcblas: 3.8.0-14_openblas conda-forge
libffi: 3.2.1-h4c5d2ac_1006 conda-forge
libgcc-ng: 7.3.0-h5c90dd9_0 c4aarch64
libgfortran-ng: 7.3.0-h6bc79d0_0 c4aarch64
liblapack: 3.8.0-14_openblas conda-forge
libopenblas: 0.3.7-h5ec1e0e_4 conda-forge
libstdcxx-ng: 7.3.0-h5c90dd9_0 c4aarch64
more-itertools: 8.0.2-py_0 conda-forge
ncurses: 6.1-hf484d3e_1002 conda-forge
numpy: 1.17.3-py38h91f3968_0 conda-forge
openssl: 1.1.1d-h516909a_0 conda-forge
packaging: 19.2-py_0 conda-forge
pluggy: 0.13.1-py38_0 conda-forge
py: 1.8.0-py_0 conda-forge
pyparsing: 2.4.5-py_0 conda-forge
pytest: 5.3.1-py38_0 conda-forge
pytest-forked: 1.1.2-py_0 conda-forge
pytest-sugar: 0.9.2-py_0 conda-forge
pytest-timeout: 1.3.3-py_0 conda-forge
pytest-xdist: 1.30.0-py_0 conda-forge
python: 3.8.0-heaf0f07_5 conda-forge
readline: 8.0-h75b48e3_0 conda-forge
scikit-learn: 0.22-py38h1971d64_0 local
scipy: 1.3.2-py38hb5cb654_0 conda-forge
setuptools: 42.0.2-py38_0 conda-forge
six: 1.13.0-py38_0 conda-forge
sqlite: 3.30.1-h283c62a_0 conda-forge
termcolor: 1.1.0-py_2 conda-forge
tk: 8.6.10-hed695b0_0 conda-forge
wcwidth: 0.1.7-py_1 conda-forge
xz: 5.2.4-hda93590_1001 conda-forge
zipp: 0.6.0-py_0 conda-forge
zlib: 1.2.11-h516909a_1006 conda-forge
https://cloud.drone.io/conda-forge/scikit-learn-feedstock/32/3/2
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with sklearn/manifold/tests/test_t_sne.py and run test_uniform_grid[barnes_hut] on aarch64 using the environment described in the report. Investigate the platform-dependent numerical failure and verify that the test passes reliably on aarch64 without regressing the other parametrized cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python, scikit-learn
- Domain
- machine-learning, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100