explosion / explosion/spaCy

Lookaround operators on Matcher patterns

Open
#6,420 3 comments 2 reactions 0 assignees View on GitHub
enhancement feat / matcher help wanted
Dominant language
Python
Stars
33.9k
Forks
4.7k
Avg merge
3m
Merged PRs (30d)
1

Description

## Feature description

The [Matcher](https://spacy.io/api/matcher) supports `!`, `?`, `+`, and `*` [operators and quantifiers](https://spacy.io/usage/rule-based-matching#quantifiers). I have text where it would be useful to have something like the regex lookaround patterns, where a pattern should or should not be matched, but is not included as part of the matched range.

For example, consider the following text.

```
Haul from AB CD site to XY site.
```

I want to create patterns for `AB CD site` and `XY site` and label them as source and destination spans. The `from` and `to` tokens are needed to distinguish between `AB CD site` and `XY site`, but should not be part of the match.

```python
from spacy.lang.en import English
from spacy.matcher import Matcher

nlp = English()

m = Matcher(nlp.vocab)
m.add("from_loc", None, [{"ORTH": "from"}, {"ORTH": {"NOT_IN": ["to"]}, "OP": "+"}, {"ORTH": "site"}])
m.add("to_loc", None, [{"ORTH": "to"}, {"ORTH": {"NOT_IN": ["from"]}, "OP": "+"}, {"ORTH": "site"}])

doc = nlp.make_doc("Haul from AB CD site to XY site.")
matches = m(doc)

for match_id, start, end in matches:
print(doc[start:end])
```

```
from AB CD site
to XY site
```

The first match span the tokens for `from AB CD site`. I want just `AB CD site` back as the match. Same for the second match.

## Proposal

The Matcher should support the following new ops, roughly based on the regex counterparts.

| Op | Name | Description |
|---|---|---|
| `?=` | Positive lookaround | The token pattern matches, but is not part of the match result. |
| `?!` | Negative lookaround | The token pattern does not match, and is not part of the match result. |

Zero or more lookaround can be used as the start and end of the pattern. A lookaround operator cannot be surrounded on both sides by non-lookaround operators in a pattern.

While there is a distinction between lookahead and lookbehind in regex, these operators are just positive/negative matchers that are not included in the result.

```python
m = Matcher(nlp.vocab)
m.add("from_loc", None, [{"ORTH": "from", "OP": "?="}, {"ORTH": {"NOT_IN": ["to"]}, "OP": "+"}, {"ORTH": "site"}])
m.add("to_loc", None, [{"ORTH": "to", "OP": "?="}, {"ORTH": {"NOT_IN": ["from"]}, "OP": "+"}, {"ORTH": "site"}])

doc = nlp.make_doc("Haul from AB CD site to XY site.")
matches = m(doc)

for match_id, start, end in matches:
print(doc[start:end])
```

```
AB CD site
XY site
```

The `from` and `to` tokens are matched by not part of the match range.

## Could the feature be a [custom component](https://spacy.io/usage/processing-pipelines#custom-components) or [spaCy plugin](https://spacy.io/universe)?

No.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.