On disk FST objects [LUCENE-5174]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
If one wants to support multiple language suggestions at the same time via AnalyzingSuggester/FuzzySuggester on Android, it's almost not possible for the time being, because all suggesters use in memory resident FST's. And of course each language needs its own FST. On Android there are VM memory restrictions of 32MB for older devices like the Nexus S. Making the math: a good language FST is roughly 11-15MB in size. Supporting even 2 languages at the same time is therefore difficult taking into account that FST's are not the only part of a common Android app.
A possible approach to a solution via memory mapping and DirectByteBuffer has been proposed by Mike Mc Candless on Lucene ML:
---
Migrated from [LUCENE-5174](https://issues.apache.org/jira/browse/LUCENE-5174) by Anna Björk Nikulásdóttir
Contributor guide
Research direction
The issue names no source files, tests, or entry points. Start by reading the linked Lucene mailing-list proposal and tracing the AnalyzingSuggester and FuzzySuggester FST handling. Done should mean supporting on-disk or memory-mapped FSTs while allowing multiple language suggesters within Android memory limits.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, java
- Domain
- mobile-dev, search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100