stored field retrieve slow [LUCENE-2252]

IndexReader.document() on a stored field is rather slow. Did a simple multi-threaded test and profiled it:

40+% time is spent in getting the offset from the index file
30+% time is spent in reading the count (e.g. number of fields to load)

Although I ran it on my lap top where the disk isn't that great, but still seems to be much room in improvement, e.g. load field index file into memory (for a 5M doc index, the extra memory footprint is 20MB, peanuts comparing to other stuff being loaded)

A related note, are there plans to have custom segments as part of flexible indexing feature?

---
Migrated from [LUCENE-2252](https://issues.apache.org/jira/browse/LUCENE-2252) by John Wang, updated Mar 24 2010

Start at IndexReader.document() and reproduce the reported stored-field retrieval slowdown with a representative index and multithreaded workload. Profile offset lookup and field-count reading as described, then compare potential improvements against memory use and verify that stored-field retrieval remains correct and measurably faster.

Written by the indexing model from the issue text.