Currently Lucene doesn't have an analyzer for Sinhala. We have built analyzer which consist of language dependent tokenizer, stemming algorithm and list of stop words. [LUCENE-9043]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
This component is developed based on three main researches.
Sinhala Analyzer, as it word implies it is an enhanced software library to analyze documents which are written in Sinhala language. Sinhala Analyzer has implemented by performing Sinhala morphological analysis. Tokenizing the document content precisely, Removing stopwords accordingly and converting the terms to its base/root form accurately are the main three functionalities of Sinhala Analyzer.
---
Migrated from [LUCENE-9043](https://issues.apache.org/jira/browse/LUCENE-9043) by pavithra kariyawasam, updated Feb 26 2020
Attachments: [SinhalaAnalyzer.java](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-9043/SinhalaAnalyzer.java), [SinhalaStemmer.java](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-9043/SinhalaStemmer.java), [SinhalaTokenizer.java](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-9043/SinhalaTokenizer.java), [stopwords.txt](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-9043/stopwords.txt)
Linked issues:
- #10086
Contributor guide
Research direction
Start by reading the attached SinhalaAnalyzer.java, SinhalaStemmer.java, SinhalaTokenizer.java, and stopwords.txt to understand the proposed analyzer and its three research-based components. Compare them with existing Lucene analyzers and identify the integration and testing points; done means Sinhala text is tokenized, stop words are removed, and terms are reduced to their root form.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100