russian pos tag mapping is wrong
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.8k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
Current mapping is
! .
A ADJ
AD ADV
C CONJ
COMP CONJ
IJ X
NC NUM
NN NOUN
P ADP
PTCL PRT
V VERB
VG VERB
VI VERB
VP VERB
YES_NO_SENT X
Z X
while on http://www.ruscorpora.ru/en/corpora-morph.html it is different.
For example,
>>> pos_tag(word_tokenize("Илья оторопел и дважды перечитал бумажку."), lang='rus')
[('Илья', 'S'), ('оторопел', 'V'), ('и', 'CONJ'), ('дважды', 'ADV'), ('перечитал', 'V'),
('бумажку', 'S'), ('.', 'NONLEX')]
which is taken from https://www.nltk.org/api/nltk.tag.html
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the Russian POS-tag mapping used by pos_tag(..., lang='rus') and compare it with the Russian National Corpus reference linked in the issue. Update the mapping to match the documented tags, then validate it against the provided tokenization and tagging example.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100