MaartenGr / MaartenGr/KeyBERT

Newer versions of Spacy transformer model backends failing

Open
#246 8 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.2k
Forks
385
PR merge metrics
No merged PRs in 30d

Description

I use spacy's transformer model for other purposes (such as NER), so re-using the same model made sense.
Looks like Spacy made some tweaks to their syntax which are breaking KeyBERT's spacy backend.

Sample code:
```
from keybert import KeyBERT
from spacy import load

nlp = load("en_core_web_trf", exclude=['tagger', 'parser', 'ner', 'attribute_ruler', 'lemmatizer'])
kw_model = KeyBERT(model=nlp)

text = "This is a test sentence."

keywords = kw_model.extract_keywords(text, keyphrase_ngram_range=(1, 1), stop_words='english', top_n=1, use_mmr=True)
print(keywords)

```
Expected behavior:
prints [("test", ...)]

Observed behavior:

```
Traceback (most recent call last):
File "...\anaconda3\envs\env\lib\site-packages\keybert\backend\_spacy.py", line 84, in embed
self.embedding_model(doc)._.trf_data.tensors[-1][0].tolist()
AttributeError: 'DocTransformerOutput' object has no attribute 'tensors'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "...\test.py", line 9, in
keywords = kw_model.extract_keywords(text, keyphrase_ngram_range=(1, 1), stop_words='english', top_n=1, use_mmr=True)
File "...\envs\env\lib\site-packages\keybert\_model.py", line 195, in extract_keywords
doc_embeddings = self.model.embed(docs)
File "...\envs\env\lib\site-packages\keybert\backend\_spacy.py", line 88, in embed
self.embedding_model("An empty document")
AttributeError: 'DocTransformerOutput' object has no attribute 'tensors'
```

Package versions:
cupy-cuda11x 12.3.0
curated-tokenizers 0.0.9
curated-transformers 0.1.1
en-core-web-trf 3.7.3
keybert 0.8.5
keyphrase-vectorizers 0.0.13
safetensors 0.4.4
scikit-learn 1.5.1
scipy 1.13.1
sentence-transformers 3.0.1
spacy 3.7.5
spacy-alignments 0.9.1
spacy-curated-transformers 0.2.2
spacy-legacy 3.0.12
spacy-loggers 1.0.5
spacy-transformers 1.3.5
thinc 8.2.5
tokenizers 0.15.2
transformers 4.36.2

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with keybert/backend/_spacy.py and the KeyBERT.extract_keywords path, then reproduce the sample with the listed spaCy 3.7.5 and transformer packages. Verify that the spaCy backend handles the current transformer output and that the example returns the expected keyword without the reported AttributeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.