de_dep_news_trf uses 1990s Spelling Convention in Lemmatization
Open
feat / lemmatizer
lang / de
- Dominant language
- Python
- Stars
- 33.9k
- Forks
- 4.7k
- Avg merge
- 3m
- Merged PRs (30d)
- 1
Description
## How to reproduce the behaviour
```
import spacy
nlp = spacy.load("de_dep_news_trf")
assert nlp("Du ißt Äpfel")[1].lemma_ == 'essen'
print(nlp("Du isst Äpfel")[1].lemma_)
```
This prints `isst` where `essen` would be expected.
Looks like the model just uses a lookup table which doesn't contain the 1996 changes to German spelling conventions. Same effect is observable for `frißt/frisst` as well.
## Your Environment
de-dep-news-trf @ https://github.com/explosion/spacy-models/releases/download/de_dep_news_trf-3.2.0/de_dep_news_trf-3.2.0-py3-none-any.whl
spacy==3.2.0
spacy-alignments==0.8.4
spacy-legacy==3.0.8
spacy-loggers==1.0.1
spacy-transformers==1.1.2
Contributor guide
Assessment
This issue has not been assessed yet.