MaartenGr / MaartenGr/PolyFuzz
from polyfuzz import PolyFuzz
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 803
- Forks
- 72
- PR merge metrics
- No merged PRs in 30d
Description
Although I was able to use PolyFuzz once for some of your basic example code, once I tried messing around with Embeddings or Bert, the entire package broke. It seems to have to do with differing numpy version compatibilities. Currently, if I do a basic
`pip install polyfuzz
`
followed by
`from polyfuzz import PolyFuzz
`
I get the following error.
```
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Input In [63], in ()
----> 1 from polyfuzz import PolyFuzz
File /opt/conda/envs/vespid/lib/python3.9/site-packages/polyfuzz/__init__.py:1, in
----> 1 from .polyfuzz import PolyFuzz
2 __version__ = "0.3.2"
File /opt/conda/envs/vespid/lib/python3.9/site-packages/polyfuzz/polyfuzz.py:7, in
5 from polyfuzz.linkage import single_linkage
6 from polyfuzz.utils import check_matches, check_grouped, create_logger
----> 7 from polyfuzz.models import TFIDF, RapidFuzz, Embeddings, BaseMatcher
8 from polyfuzz.metrics import precision_recall_curve, visualize_precision_recall
10 logger = create_logger()
File /opt/conda/envs/vespid/lib/python3.9/site-packages/polyfuzz/models/__init__.py:4, in
2 from ._distance import EditDistance
3 from ._rapidfuzz import RapidFuzz
----> 4 from ._tfidf import TFIDF
5 from ._utils import cosine_similarity
7 from polyfuzz.error import NotInstalled
File /opt/conda/envs/vespid/lib/python3.9/site-packages/polyfuzz/models/_tfidf.py:7, in
4 from typing import List, Tuple
5 from sklearn.feature_extraction.text import TfidfVectorizer
----> 7 from ._utils import cosine_similarity
8 from ._base import BaseMatcher
11 class TFIDF(BaseMatcher):
File /opt/conda/envs/vespid/lib/python3.9/site-packages/polyfuzz/models/_utils.py:9, in
6 from sklearn.metrics.pairwise import cosine_similarity as scikit_cosine_similarity
8 try:
----> 9 from sparse_dot_topn import awesome_cossim_topn
10 _HAVE_SPARSE_DOT = True
11 except ImportError:
File /opt/conda/envs/vespid/lib/python3.9/site-packages/sparse_dot_topn/__init__.py:5, in
2 import sys
4 if sys.version_info[0] >= 3:
----> 5 from sparse_dot_topn.awesome_cossim_topn import awesome_cossim_topn
6 else:
7 from awesome_cossim_topn import awesome_cossim_topn
File /opt/conda/envs/vespid/lib/python3.9/site-packages/sparse_dot_topn/awesome_cossim_topn.py:7, in
4 from scipy.sparse import isspmatrix_csr
6 if sys.version_info[0] >= 3:
----> 7 from sparse_dot_topn import sparse_dot_topn as ct
8 from sparse_dot_topn import sparse_dot_topn_threaded as ct_thread
9 else:
File /opt/conda/envs/vespid/lib/python3.9/site-packages/sparse_dot_topn/sparse_dot_topn.pyx:1, in init sparse_dot_topn.sparse_dot_topn()
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
```
Following some StackOverflow posts, I tried installing differing versions of numpy, but in the end, *something* is always unhappy, and somehow I can no longer use PolyFuzz no matter what I do. It would be great if it would work with the latest version of numpy, or if at least one version *definitely* worked reliably! Thanks for looking into this.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the reported `pip install polyfuzz` followed by `from polyfuzz import PolyFuzz` failure, then trace the imports through `polyfuzz/models/_utils.py` and `sparse_dot_topn`. Check the NumPy and related dependency versions involved; done means a documented supported combination or latest-NumPy installation can import PolyFuzz without the binary incompatibility error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python, scikit-learn
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100