add NGramSynonymTokenizer [LUCENE-5252]

I'd like to propose that we have another n-gram tokenizer which can process synonyms. That is NGramSynonymTokenizer. Note that in this ticket, the gram size is fixed, i.e. minGramSize = maxGramSize.

Today, I think we have the following problems when using SynonymFilter with NGramTokenizer.
For purpose of illustration, we have a synonym setting "ABC, DEFG" w/ expand=true and N = 2 (2-gram).

1. There is no consensus (I think :-) how we assign offsets to generated synonym tokens DE, EF and FG when expanding source token AB and BC.
1. If the query pattern looks like ABCY, it cannot be matched even if there is a document "…ABCY…" in index when autoGeneratePhraseQueries set to true, because there is no "CY" token (but "GY" is there) in the index.

NGramSynonymTokenizer can solve these problems by providing the following methods.

- NGramSynonymTokenizer reads synonym settings (synonyms.txt) and it doesn't tokenize registered words. e.g.

|source text|NGramTokenizer+SynonymFilter|NGramSynonymTokenizer|
|-|-|-|
|ABC|AB/DE/BC/EF/FG|ABC/DEFG|

- The back and forth of the registered words, NGramSynonymTokenizer generates **extra** tokens w/ posInc=0. e.g.

|source text|NGramTokenizer+SynonymFilter|NGramSynonymTokenizer|
|-|-|-|
|XYZABC123|XY/YZ/ZA/AB/DE/BC/EF/C1/FG/12/23|XY/YZ/Z/ABC/DEFG/1/12/23|

In the above sample, "Z" and "1" are the extra tokens.

---
Migrated from [LUCENE-5252](https://issues.apache.org/jira/browse/LUCENE-5252) by Koji Sekiguchi (@kojisekig), 1 vote, updated Oct 18 2013
Attachments: [LUCENE-5252_4x.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-5252/LUCENE-5252_4x.patch) (versions: 5)
Linked issues:
- #6317

Start by reading the existing NGramTokenizer and SynonymFilter behavior, then review the attached LUCENE-5252_4x.patch and the synonym examples in this issue. A complete change would add NGramSynonymTokenizer with fixed gram sizes, synonym-aware token handling, and the described extra tokens and position increments.

Written by the indexing model from the issue text.