ShingleFilter should have an option to skip filler tokens (e.g. stop words) [LUCENE-8036]

ShingleFilterFactory should have an option to ignore filler tokens in the total shingle size.
For instance (adapted from ), consider the text "A brown fox quickly jumps over the lazy dog". When we remove stopwords and execute the ShingleFilter (shingle size = 3), it gives us the following result:

1. _ brown fox
2. brown fox quickly
3. fox quickly jump
4. quickly jump _
5. jump _ _
6. _ _ lazy
7. _ lazy dog

We can clearly see that the filler token "_" occupies one token in the shingle.
I suppose the returned shingles should be:
1. brown fox quickly
2. fox quickly jump
3. quickly jump lazy
4. jump lazy dog

To maintain backward compatibility, i suggest the creation of an option called "skipFillerTokens" to implement this behavior (note that this is different than using fillerTokens="", since the empty string occupies one token in the shingle)

I've attached a patch for the ShingleFilter class (getNextToken() method), ShingleFilterFactory and ShingleFilterTest clases.

---
Migrated from [LUCENE-8036](https://issues.apache.org/jira/browse/LUCENE-8036) by Edans Sandes, updated Nov 29 2017
Attachments: [SOLR-11604.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-8036/SOLR-11604.patch)
Linked issues:
- [SOLR-11605](https://issues.apache.org/jira/browse/SOLR-11605)
- [SOLR-6468](https://issues.apache.org/jira/browse/SOLR-6468)

Review the attached SOLR-11604.patch, then read ShingleFilter.getNextToken(), ShingleFilterFactory, and ShingleFilterTest. Verify how filler tokens affect shingle size and how the proposed skipFillerTokens option should preserve existing behavior by default. Done means the option produces shingles that omit filler tokens and the relevant tests cover both modes.

Written by the indexing model from the issue text.