MemoryPostings fst encoding can be surprisingly inefficient (especially in tests, with payloads) [LUCENE-6383]

I just worked around this in 2 nightly OOM fails.

One was TestDuelingCodecs, the other was TestIndexWriterForceMerge's space usage test.

In general the trend is the same, it seems the more documents you merge, you just get bigger and bigger FST outputs and the size of this PF in ram and on disk grows in a way you don't expect. E.g. merging 300KB of segments resulted in 450KB single segment, and memory usage gets absurdly high.

The issue seems especially aggravated in tests, when MockAnalyzer adds lots of payloads.

Maybe it should encode the postings data in a more efficient way? Can it just use a Long output pointing into a RAMFile or something? Or maybe there is just a crazy bug?

---
Migrated from [LUCENE-6383](https://issues.apache.org/jira/browse/LUCENE-6383) by Robert Muir (@rmuir)

Start by reproducing the failures in TestDuelingCodecs and TestIndexWriterForceMerge's space usage test. Inspect MemoryPostings FST encoding and how MockAnalyzer payloads affect postings data, then compare FST and PF memory or disk growth as segments are merged. Done means the unexpected growth is explained and the affected tests no longer exhibit the reported excessive usage.

Written by the indexing model from the issue text.