IndexError E040 when using senter
- Dominant language
- Python
- Stars
- 33.9k
- Forks
- 4.7k
- Avg merge
- 3m
- Merged PRs (30d)
- 1
Description
## How to reproduce the behaviour
As suggested on this page (https://spacy.io/models) I can replace the "parser" by the "senter" as a more efficient way to detect sentence boundaries. I did so, but ran into an E040 error when printing the individual noun_chunks of the document (which is not happening when using the original parser).
Code to reproduce:
```
nlp = spacy.load("en_core_web_sm")
nlp.disable_pipe('parser')
nlp.enable_pipe('senter')
doc = nlp(text)
for chunk in doc.noun_chunks:
print(chunk)
```
Error message:
```
File "spacy\tokens\token.pyx", line 609, in spacy.tokens.token.Token.left_edge.__get__
File "spacy\tokens\doc.pyx", line 474, in spacy.tokens.doc.Doc.__getitem__
File "spacy\tokens\token.pxd", line 23, in spacy.tokens.token.Token.cinit
IndexError: [E040] Attempt to access token at 10794, max length 10792.
```
## Your Environment
- **spaCy version:** 3.2.3
- **Platform:** Windows-10-10.0.19042-SP0
- **Python version:** 3.9.7
- **Pipelines:** en_core_web_lg (3.2.0), en_core_web_md (3.2.0), en_core_web_sm (3.2.0), en_core_web_trf (3.2.0)
Contributor guide
Assessment
This issue has not been assessed yet.