CentreForDigitalHumanities / CentreForDigitalHumanities/Textcavator
Add lemmatized data to ES index
- Dominant language
- Python
- Stars
- 12
- Forks
- 3
- Avg merge
- 3d 3m
- Merged PRs (30d)
- 9
Description
**Is your feature request related to a problem? Please describe.**
For the word models of Finnish parliamentary data, the preprocessing included lemmatization with spaCy's `fi_core_news_lg` model. Researchers are now wondering whether we could also have the lemmatized data available in Elasticsearch.
**Describe the solution you'd like**
We can arguably add another subfield of `speech` and add lemmatized data. Unfortunately, the lemmatized data from the word embedding preprocessing cannot be reused, as this chunks all speeches from one year together.
**Alternative solutions**
Leave analysis of lemmatized data out of the scope of I-Analyzer - we could also suggest that users need to roll their own.
**Problems**
We would not be able to apply lemmatization to the search query as well.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.