MaartenGr / MaartenGr/PolyFuzz

from polyfuzz import PolyFuzz

Open
#36 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
803
Forks
72
PR merge metrics
No merged PRs in 30d

Description

Although I was able to use PolyFuzz once for some of your basic example code, once I tried messing around with Embeddings or Bert, the entire package broke. It seems to have to do with differing numpy version compatibilities. Currently, if I do a basic

`pip install polyfuzz
`
followed by

`from polyfuzz import PolyFuzz
`
I get the following error.

```
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Input In [63], in ()
----> 1 from polyfuzz import PolyFuzz

File /opt/conda/envs/vespid/lib/python3.9/site-packages/polyfuzz/__init__.py:1, in
----> 1 from .polyfuzz import PolyFuzz
2 __version__ = "0.3.2"

File /opt/conda/envs/vespid/lib/python3.9/site-packages/polyfuzz/polyfuzz.py:7, in
5 from polyfuzz.linkage import single_linkage
6 from polyfuzz.utils import check_matches, check_grouped, create_logger
----> 7 from polyfuzz.models import TFIDF, RapidFuzz, Embeddings, BaseMatcher
8 from polyfuzz.metrics import precision_recall_curve, visualize_precision_recall
10 logger = create_logger()

File /opt/conda/envs/vespid/lib/python3.9/site-packages/polyfuzz/models/__init__.py:4, in
2 from ._distance import EditDistance
3 from ._rapidfuzz import RapidFuzz
----> 4 from ._tfidf import TFIDF
5 from ._utils import cosine_similarity
7 from polyfuzz.error import NotInstalled

File /opt/conda/envs/vespid/lib/python3.9/site-packages/polyfuzz/models/_tfidf.py:7, in
4 from typing import List, Tuple
5 from sklearn.feature_extraction.text import TfidfVectorizer
----> 7 from ._utils import cosine_similarity
8 from ._base import BaseMatcher
11 class TFIDF(BaseMatcher):

File /opt/conda/envs/vespid/lib/python3.9/site-packages/polyfuzz/models/_utils.py:9, in
6 from sklearn.metrics.pairwise import cosine_similarity as scikit_cosine_similarity
8 try:
----> 9 from sparse_dot_topn import awesome_cossim_topn
10 _HAVE_SPARSE_DOT = True
11 except ImportError:

File /opt/conda/envs/vespid/lib/python3.9/site-packages/sparse_dot_topn/__init__.py:5, in
2 import sys
4 if sys.version_info[0] >= 3:
----> 5 from sparse_dot_topn.awesome_cossim_topn import awesome_cossim_topn
6 else:
7 from awesome_cossim_topn import awesome_cossim_topn

File /opt/conda/envs/vespid/lib/python3.9/site-packages/sparse_dot_topn/awesome_cossim_topn.py:7, in
4 from scipy.sparse import isspmatrix_csr
6 if sys.version_info[0] >= 3:
----> 7 from sparse_dot_topn import sparse_dot_topn as ct
8 from sparse_dot_topn import sparse_dot_topn_threaded as ct_thread
9 else:

File /opt/conda/envs/vespid/lib/python3.9/site-packages/sparse_dot_topn/sparse_dot_topn.pyx:1, in init sparse_dot_topn.sparse_dot_topn()

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
```

Following some StackOverflow posts, I tried installing differing versions of numpy, but in the end, *something* is always unhappy, and somehow I can no longer use PolyFuzz no matter what I do. It would be great if it would work with the latest version of numpy, or if at least one version *definitely* worked reliably! Thanks for looking into this.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the reported `pip install polyfuzz` followed by `from polyfuzz import PolyFuzz` failure, then trace the imports through `polyfuzz/models/_utils.py` and `sparse_dot_topn`. Check the NumPy and related dependency versions involved; done means a documented supported combination or latest-NumPy installation can import PolyFuzz without the binary incompatibility error.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python, scikit-learn
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.