Parser doesn't respect preset sentence boundaries in some cases
- Dominant language
- Python
- Stars
- 33.9k
- Forks
- 4.7k
- Avg merge
- 3m
- Merged PRs (30d)
- 1
Description
This is for the issue found in https://github.com/explosion/spaCy/discussions/7564.
## How to reproduce the behaviour
Given a sentence, set `is_sent_start` to False in some but not all of the tokens before it gets to the parser.
Example sentence:
> According to the sections (1) and (2) of this page.
Example settings:
```
According True
to None
the None
sections None
( True
1 None
) None
and False
( False
2 None
) None
of False
this None
page None
. None
```
In this case the parser will contradict the False settings. However, if you move them one down to the next token, they will be respected. This seems like an off-by-one error in the parser somewhere.
## Your Environment
* Operating System: Linux
* Python Version Used:
* spaCy Version Used: 3.0.5
* Environment Information:
Contributor guide
Assessment
This issue has not been assessed yet.