lemmatization errors German verb 'sein'
Open
Nobody has claimed this yet.
review
- Dominant language
- Python
- Stars
- 8.9k
- Forks
- 1.6k
- PR merge metrics
- No merged PRs in 30d
Description
@CJAnti reported a pattern related issue to textblob-de which uses a Python3 compatible version of pattern.de or the original pattern distribution (if installed) on Python2.
https://github.com/markuskiller/textblob-de/issues/9
EXPECTED:
Du
bist --> Lemma: sein
Ihr
seid --> Lemma: sein
# Tested on Python2.7.8, 32bit, on Windows 8.1 (64bit)
# pattern.__version__
# '2.6'
In [1]: from pattern.de import parse, pprint
In [2]: pprint(parse("Ich bin. Du bist. Er ist. Wir sind. Ihr seid. Sie sind.", lemmata=True))
WORD TAG CHUNK ROLE ID PNP LEMMA
Ich PRP NP - - - ich
bin VB VP - - - sein
. . - - - - .
WORD TAG CHUNK ROLE ID PNP LEMMA
Du PRP NP - - - du
bist NN NP ^ - - - bist
. . - - - - .
WORD TAG CHUNK ROLE ID PNP LEMMA
Er PRP NP - - - er
ist VB VP - - - sein
. . - - - - .
WORD TAG CHUNK ROLE ID PNP LEMMA
Wir PRP NP - - - wir
sind VB VP - - - sein
. . - - - - .
WORD TAG CHUNK ROLE ID PNP LEMMA
Ihr PRP$ NP - - - ihr
seid NN NP ^ - - - seid
. . - - - - .
WORD TAG CHUNK ROLE ID PNP LEMMA
Sie PRP NP - - - sie
sind VB VP - - - sein
. . - - - - .
In [3]: pprint(parse("Ihr seid alle herzlich eingeladen zu meinem Geburtstagsfest.", lemmata=True))
WORD TAG CHUNK ROLE ID PNP LEMMA
Ihr PRP$ NP - - - ihr
seid NN NP ^ - - - seid
alle RB ADJP - - - alle
herzlich JJ ADJP ^ - - - herzlich
eingeladen VBN VP - - - einladen
zu IN PP - - PNP zu
meinem PRP$ NP - - PNP meinem
Geburtstagsfest NN NP ^ - - PNP geburtstagsfest
. . - - - - .
In [4]: pprint(parse("Du bist herzlich eingeladen zu meinem Geburtstagsfest.", lemmata=True))
WORD TAG CHUNK ROLE ID PNP LEMMA
Du PRP NP - - - du
bist NN NP ^ - - - bist
herzlich JJ ADJP - - - herzlich
eingeladen VBN VP - - - einladen
zu IN PP - - PNP zu
meinem PRP$ NP - - PNP meinem
Geburtstagsfest NN NP ^ - - PNP geburtstagsfest
. . - - - - .
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the examples with pattern.de.parse(..., lemmata=True) and inspect the output from pprint. The issue is resolved when German “bist” and “seid” are tagged and lemmatized consistently as “sein” in the reported sentence contexts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100