explosion / explosion/spaCy

Parser doesn't respect preset sentence boundaries in some cases

Open
#7,716 7 comments 0 reactions 0 assignees View on GitHub
bug feat / parser
Dominant language
Python
Stars
33.9k
Forks
4.7k
Avg merge
3m
Merged PRs (30d)
1

Description

This is for the issue found in https://github.com/explosion/spaCy/discussions/7564.

## How to reproduce the behaviour

Given a sentence, set `is_sent_start` to False in some but not all of the tokens before it gets to the parser.

Example sentence:

> According to the sections (1) and (2) of this page.

Example settings:

```
According True
to None
the None
sections None
( True
1 None
) None
and False
( False
2 None
) None
of False
this None
page None
. None
```

In this case the parser will contradict the False settings. However, if you move them one down to the next token, they will be respected. This seems like an off-by-one error in the parser somewhere.

## Your Environment

* Operating System: Linux
* Python Version Used:
* spaCy Version Used: 3.0.5
* Environment Information:

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.