Currently Lucene doesn't have an analyzer for Sinhala. We have built analyzer which consist of language dependent tokenizer, stemming algorithm and list of stop words. [LUCENE-9044]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
This component is developed based on three main researches.
Lucene did not have component to analyze Sinhala documents. So our intension is to fill that space with an Analyzer which can analyze Sinhala documents. Sinhala Analyzer has implemented by performing Sinhala morphological analysis. Tokenizing the document content precisely, Removing stopwords accordingly and converting the terms to its base/root form accurately are the main three functionalities of Sinhala Analyzer. These are built by considering the grammatical rules in Sinhala
---
Migrated from [LUCENE-9044](https://issues.apache.org/jira/browse/LUCENE-9044) by pavithra kariyawasam, 1 vote, updated Feb 26 2020
Environment:
```
Lucene
```
Attachments: [SinhalaAnalyzer.java](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-9044/SinhalaAnalyzer.java), [SinhalaStemmer.java](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-9044/SinhalaStemmer.java), [SinhalaTokenizer.java](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-9044/SinhalaTokenizer.java), [stopwords.txt](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-9044/stopwords.txt)
Linked issues:
- #10085
Contributor guide
Research direction
Start by reviewing the attached SinhalaAnalyzer.java, SinhalaStemmer.java, SinhalaTokenizer.java, and stopwords.txt, then inspect linked issue #10085 for surrounding context. Determine how the proposed analyzer should fit into Lucene and verify tokenization, stopword removal, and stemming behavior; done means the Sinhala analysis component is integrated and covered by appropriate tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100