Make graph-based TokenFilters easier [LUCENE-5012]

SynonymFilter has two limitations today:

- It cannot create positions, so eg dns -> domain name service
creates blatantly wrong highlights (SOLR-3390, #5565 and
others).

- It cannot consume a graph, so e.g. if you try to apply synonyms
after Kuromoji tokenizer I'm not sure what will happen.

I've thought about how to fix these issues but it's really quite
difficult with the current PosInc/PosLen graph representation, so I'd
like to explore an alternative approach.

---
Migrated from [LUCENE-5012](https://issues.apache.org/jira/browse/LUCENE-5012) by Michael McCandless (@mikemccand), 5 votes, updated Dec 17 2017
Attachments: [LUCENE-5012.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-5012/LUCENE-5012.patch) (versions: 2)